kubernetes k8s CRD自定义资源学习笔记
时间：2023-02-25 11:45:02 | 栏目： | 点击：次
CustomResourceDefinition简介：
在 Kubernetes 中一切都可视为资源，Kubernetes 1.7 之后增加了对 CRD 自定义资源二次开发能力来扩展 Kubernetes API，通过 CRD 我们可以向 Kubernetes API 中增加新资源类型，而不需要修改 Kubernetes 源码来创建自定义的 API server，该功能大大提高了 Kubernetes 的扩展能力。
当你创建一个新的CustomResourceDefinition (CRD)时，Kubernetes API服务器将为你指定的每个版本创建一个新的RESTful资源路径，我们可以根据该api路径来创建一些我们自己定义的类型资源。CRD可以是命名空间的，也可以是集群范围的，由CRD的作用域(scpoe)字段中所指定的，与现有的内置对象一样，删除名称空间将删除该名称空间中的所有自定义对象。customresourcedefinition本身没有名称空间，所有名称空间都可以使用。
目前扩展Kubernetes API的常用方式有3种:
- 使用CRD(CustomResourceDefinitions)自定义资源类型
- 开发自定义的APIServer并聚合至主API Server
- 及定制扩展API Server源码。这其中，CRD最为易用但限制颇多，自定义API Server更富于弹性但代码工作量偏大，而仅在必须添加新的核心类型才能确保专用的Kberneves集群功能正常，才应该定制系统源码
CRD-->CRT-->CR
- 其中CRD与CRT一般由开发或服务供应商提供
- CRD只是定义一个类型Kind，但实际把kind运行起来CR需要有Controller来对资源进行控制，所有只有定义CRD定义没有并没有实际意义，当然也可以通过定义现在kind来运行，比如deployment 通过定义 RC来运行
配置规范
apiVersion: apiextensions.k8s.io/v1 #API群组和版本
kind: CustomResourceDefinition #资源类别
metadata:
-name <string> #资源名称
spec:
conversion <object> #定义不同版本间的格式转换方式
strategy <string># 不同版本间的自定义资源转换策略，有None和webhook两种取值
webhook <0bject>#如何调用用于进行格式转换的webhook
group <string>#资源所属的API群组
names <object># 自定义资源的类型，即该CRD创建资源规范时使用的kind
categories <[]string>#资源所属的类别编目，例如"kubectl get all"中的all
kind <string> #kind名称,必选字段
listKind <string> #资源列表名称，默认为"`kind`List"
plural <string> #复数，用于API路径`/apis/<group>/<version>/. . ./<plural>'
shortNames <[string>#该资源的kind的缩写格式
singular <string>#资源kind的单数形式，必须使用全小写字母，默认为小写的kind名称
preserveUnknownFields <boolean> #预留的非知名字段，kind等都是知名的预留字段
scope <string> #作用域，可用值为Cluster和Namespaced
versions <[]object>#版本号定义
additionalPrinterColumns <[]0bject> #需要返回的额外信息
name <string> #形如vM[alphaN|betaN]格式的版本名称，例如v1或vlalpha2等
schema <object> #该资源的数据格式(schema)定义，必选字段
openAPIV3Schema <object> #用于校验字段的schema对象，格式请参考相关手册
served <boolean> #是否允许通过RESTful API调度该版本，必选字段
storage <boolean> #将自定义资源存储于etcd中时是不是使用该版本
subresources <0bject>#子资源定义
scale <0bject># 启用scale子资源，通过autoscaling/v1.Scale发送负荷
status <map[string]># 启用status子资源，为资源生成/status端点
可以查看之前部署Calico创建的自定义CRD
[root@k8s-master ~]# kubectl api-resources #查看所有资源类型 NAME SHORTNAMES APIGROUP NAMESPACED KIND ... bgpconfigurations crd.projectcalico.org false BGPConfiguration bgppeers crd.projectcalico.org false BGPPeer blockaffinities crd.projectcalico.org false BlockAffinity clusterinformations crd.projectcalico.org false ClusterInformation felixconfigurations crd.projectcalico.org false FelixConfiguration globalnetworkpolicies crd.projectcalico.org false GlobalNetworkPolicy globalnetworksets crd.projectcalico.org false GlobalNetworkSet hostendpoints crd.projectcalico.org false HostEndpoint ipamblocks crd.projectcalico.org false IPAMBlock ipamconfigs crd.projectcalico.org false IPAMConfig ipamhandles crd.projectcalico.org false IPAMHandle ippools crd.projectcalico.org false IPPool kubecontrollersconfigurations crd.projectcalico.org false KubeControllersConfiguration networkpolicies crd.projectcalico.org true NetworkPolicy networksets crd.projectcalico.org true NetworkSet
查看calico的yaml文件可以看到里面很多CRD的定义
[root@k8s-master plugin]# vim calico.yaml ... --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: ippools.crd.projectcalico.org spec: ...... ... [root@k8s-master plugin]# kubectl get CustomResourceDefinition NAME CREATED AT bgpconfigurations.crd.projectcalico.org 2021-08-29T14:33:24Z bgppeers.crd.projectcalico.org 2021-08-29T14:33:24Z blockaffinities.crd.projectcalico.org 2021-08-29T14:33:24Z clusterinformations.crd.projectcalico.org 2021-08-29T14:33:24Z felixconfigurations.crd.projectcalico.org 2021-08-29T14:33:24Z globalnetworkpolicies.crd.projectcalico.org 2021-08-29T14:33:24Z globalnetworksets.crd.projectcalico.org 2021-08-29T14:33:24Z hostendpoints.crd.projectcalico.org 2021-08-29T14:33:24Z ipamblocks.crd.projectcalico.org 2021-08-29T14:33:24Z ipamconfigs.crd.projectcalico.org 2021-08-29T14:33:24Z ipamhandles.crd.projectcalico.org 2021-08-29T14:33:24Z ippools.crd.projectcalico.org 2021-08-29T14:33:24Z kubecontrollersconfigurations.crd.projectcalico.org 2021-08-29T14:33:24Z networkpolicies.crd.projectcalico.org 2021-08-29T14:33:24Z networksets.crd.projectcalico.org 2021-08-29T14:33:25Z
示例1: 创建自定义CRD
[root@k8s-master crd]# cat crd-v1-user.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: users.auth.ilinux.io
spec:
group: auth.ilinux.io
names:
kind: User
plural: users
singular: user
shortNames:
- u
scope: Namespaced #名称空间级别
versions:
- served: true
storage: true
name: v1alpha1 #版本号
schema:
openAPIV3Schema:
type: object #对字段做限制
properties:
spec:
type: object
properties:
userID:
type: integer #整形
minimum: 1
maximum: 65535
groups :
type: array #列表
items:
type: string
email:
type: string
password:
type: string
format: password
required: ["userID","groups"]
[root@k8s-master crd]# kubectl apply -f crd-v1-user.yaml
[root@k8s-master crd]# kubectl api-resources
NAME SHORTNAMES APIGROUP NAMESPACED KIND
bindings true Binding
...
users u auth.ilinux.io true User
创造自定义CRD类型
[root@k8s-master crd]# cat user-cr-demo.yaml
apiVersion: auth.ilinux.io/v1alpha1
kind: User
metadata:
name: admin
namespace: default
spec:
userID: 1
email: test@test.com
groups:
- superusers
- adminstrators
password: ikubernetes.io
[root@k8s-master crd]# kubectl apply -f user-cr-demo.yaml
user.auth.ilinux.io/admin created
[root@k8s-master crd]# kubectl get User
NAME AGE
admin 14s
[root@k8s-master ~]# kubectl describe User admin
Name: admin
Namespace: default
Labels: <none>
Annotations: <none>
API Version: auth.ilinux.io/v1alpha1
Kind: User
Metadata:
Creation Timestamp: 2021-09-10T14:51:53Z
Generation: 1
Managed Fields:
API Version: auth.ilinux.io/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:email:
f:groups:
f:password:
f:userID:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2021-09-10T14:51:53Z
Resource Version: 2583010
Self Link: /apis/auth.ilinux.io/v1alpha1/namespaces/default/users/admin
UID: 5af89454-e067-4f30-83b7-cc2ad82e3526
Spec:
Email: test@test.com
Groups:
superusers
adminstrators
Password: ikubernetes.io
User ID: 1
Events: <none>
以上定义的kind资源 没Controller并不能运行成实际对象，Controller的开发需要开发来完成
示例2: etcd Operator 部署 (该项目已不在维护)
Operator 项目地址:
https://github.com/coreos/etcd-operator/blob/master/doc/user/install_guide.md
https://github.com/coreos/etcd-operator
https://github.com/operator-framework/awesome-operators
先安装RBAC 再安装etcd operator 再部署创建etcd集群
[root@k8s-master etcd-operator]# example/rbac/create_role.sh
Creating role with ROLE_NAME=etcd-operator, NAMESPACE=default
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRole is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRole
clusterrole.rbac.authorization.k8s.io/etcd-operator created
Creating role binding with ROLE_NAME=etcd-operator, ROLE_BINDING_NAME=etcd-operator, NAMESPACE=default
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRoleBinding is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRoleBinding
clusterrolebinding.rbac.authorization.k8s.io/etcd-operator created
[root@k8s-master etcd-operator]# kubectl create -f example/deployment.yaml
error: unable to recognize "example/deployment.yaml": no matches for kind "Deployment" in version "extensions/v1beta1"
#deployment版本太老修改example/deployment.yaml
[root@k8s-master etcd-operator]# cat example/deployment.yaml
apiVersion: apps/v1 #版本
kind: Deployment
metadata:
name: etcd-operator
spec:
replicas: 1
selector: #添加字段
matchLabels:
name: etcd-operator
template:
metadata:
labels:
name: etcd-operator
spec:
containers:
- name: etcd-operator
image: quay.io/coreos/etcd-operator:v0.9.4
command:
- etcd-operator
# Uncomment to act for resources in all namespaces. More information in doc/user/clusterwide.md
#- -cluster-wide
env:
- name: MY_POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
[root@k8s-master etcd-operator]# kubectl create -f example/deployment.yaml
deployment.apps/etcd-operator created
[root@k8s-master etcd-operator]#
[root@k8s-master etcd-operator]# kubectl api-resources
...
etcdclusters etcd etcd.database.coreos.com true EtcdCluster
部署创建etcd集群
[root@k8s-master etcd-operator]# cat example/example-etcd-cluster.yaml apiVersion: "etcd.database.coreos.com/v1beta2" kind: "EtcdCluster" metadata: name: "example-etcd-cluster" ## Adding this annotation make this cluster managed by clusterwide operators ## namespaced operators ignore it # annotations: # etcd.database.coreos.com/scope: clusterwide spec: size: 3 #集群数理 version: "3.2.13" [root@k8s-master etcd-operator]# kubectl apply -f example/example-etcd-cluster.yaml etcdcluster.etcd.database.coreos.com/example-etcd-cluster created [root@k8s-master etcd-operator]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES etcd-operator-646cbffdb6-brbn6 1/1 Running 0 12m 192.168.51.58 k8s-node3 <none> <none> example-etcd-cluster-nc8pdgjrjr 1/1 Running 0 3m3s 192.168.51.59 k8s-node3 <none> <none> - 后面在加一个SVC就可以使用了