中文
插件扩展
CustomResourceDefinition
CustomResourceDefinition（CRD）是 v1.7 新增的无需改变代码就可以扩展 Kubernetes API 的机制，用来管理自定义对象。它实际上是 ThirdPartyResources（TPR）的升级版本，而 TPR 已经在 v1.8 中弃用。
API 版本对照表
Kubernetes 版本
CRD API 版本
v1.8+
apiextensions.k8s.io/v1beta1
CRD 示例
下面的例子会创建一个
/apis/stable.example.com/v1/namespaces/<namespace>/crontabs/… 的自定义 API：1
apiVersion: apiextensions.k8s.io/v1beta1
2
kind: CustomResourceDefinition
3
metadata:
4
# name must match the spec fields below, and be in the form: <plural>.<group>
5
name: crontabs.stable.example.com
6
spec:
7
# group name to use for REST API: /apis/<group>/<version>
8
group: stable.example.com
9
# versions to use for REST API: /apis/<group>/<version>
10
versions:
11
- name: v1beta1
12
# Each version can be enabled/disabled by Served flag.
13
served: true
14
# One and only one version must be marked as the storage version.
15
storage: true
16
- name: v1
17
served: true
18
storage: false
19
# either Namespaced or Cluster
20
scope: Namespaced
21
names:
22
# plural name to be used in the URL: /apis/<group>/<version>/<plural>
23
plural: crontabs
24
# singular name to be used as an alias on the CLI and for display
25
singular: crontab
26
# kind is normally the CamelCased singular type. Your resource manifests use this.
27
kind: CronTab
28
# shortNames allow shorter string to match your resource on the CLI
29
shortNames:
30
- ct
Copied!
API 创建好后，就可以创建具体的 CronTab 对象了
1
$ cat my-cronjob.yaml
2
apiVersion: "stable.example.com/v1"
3
kind: CronTab
4
metadata:
5
name: my-new-cron-object
6
spec:
7
cronSpec: "* * * * /5"
8
image: my-awesome-cron-image
9
10
$ kubectl create -f my-crontab.yaml
11
crontab "my-new-cron-object" created
12
13
$ kubectl get crontab
14
NAME KIND
15
my-new-cron-object CronTab.v1.stable.example.com
16
$ kubectl get crontab my-new-cron-object -o yaml
17
apiVersion: stable.example.com/v1
18
kind: CronTab
19
metadata:
20
creationTimestamp: 2017-07-03T19:00:56Z
21
name: my-new-cron-object
22
namespace: default
23
resourceVersion: "20630"
24
selfLink: /apis/stable.example.com/v1/namespaces/default/crontabs/my-new-cron-object
25
uid: 5c82083e-5fbd-11e7-a204-42010a8c0002
26
spec:
27
cronSpec: '* * * * /5'
28
image: my-awesome-cron-image
Copied!
Finalizer
Finalizer 用于实现控制器的异步预删除钩子，可以通过
metadata.finalizers 来指定 Finalizer。1
apiVersion: "stable.example.com/v1"
2
kind: CronTab
3
metadata:
4
finalizers:
5
- finalizer.stable.example.com
Copied!
Finalizer 指定后，客户端删除对象的操作只会设置
metadata.deletionTimestamp 而不是直接删除。这会触发正在监听 CRD 的控制器，控制器执行一些删除前的清理操作，从列表中删除自己的 finalizer，然后再重新发起一个删除操作。此时，被删除的对象才会真正删除。Validation
v1.8 开始新增了实验性的基于 OpenAPI v3 schema 的验证（Validation）机制，可以用来提前验证用户提交的资源是否符合规范。使用该功能需要配置 kube-apiserver 的
--feature-gates=CustomResourceValidation=true。比如下面的 CRD 要求
spec.cronSpec必须是匹配正则表达式的字符串spec.replicas必须是从 1 到 10 的整数
1
apiVersion: apiextensions.k8s.io/v1beta1
2
kind: CustomResourceDefinition
3
metadata:
4
name: crontabs.stable.example.com
5
spec:
6
group: stable.example.com
7
version: v1
8
scope: Namespaced
9
names:
10
plural: crontabs
11
singular: crontab
12
kind: CronTab
13
shortNames:
14
- ct
15
validation:
16
# openAPIV3Schema is the schema for validating custom objects.
17
openAPIV3Schema:
18
properties:
19
spec:
20
properties:
21
cronSpec:
22
type: string
23
pattern: '^(\d+|\*)(/\d+)?(\s+(\d+|\*)(/\d+)?){4}#x27;
24
replicas:
25
type: integer
26
minimum: 1
27
maximum: 10
Copied!
这样，在创建下面的 CronTab 时
1
apiVersion: "stable.example.com/v1"
2
kind: CronTab
3
metadata:
4
name: my-new-cron-object
5
spec:
6
cronSpec: "* * * *"
7
image: my-awesome-cron-image
8
replicas: 15
Copied!
会报验证失败的错误：
1
The CronTab "my-new-cron-object" is invalid: []: Invalid value: map[string]interface {}{"apiVersion":"stable.example.com/v1", "kind":"CronTab", "metadata":map[string]interface {}{"name":"my-new-cron-object", "namespace":"default", "deletionTimestamp":interface {}(nil), "deletionGracePeriodSeconds":(*int64)(nil), "creationTimestamp":"2017-09-05T05:20:07Z", "uid":"e14d79e7-91f9-11e7-a598-f0761cb232d1", "selfLink":"","clusterName":""}, "spec":map[string]interface {}{"cronSpec":"* * * *", "image":"my-awesome-cron-image", "replicas":15}}:
2
validation failure list:
3
spec.cronSpec in body should match '^(\d+|\*)(/\d+)?(\s+(\d+|\*)(/\d+)?){4}#x27;
4
spec.replicas in body should be less than or equal to 10
Copied!
Subresources
v1.10 开始 CRD 还支持
/status 和 /scale 等两个子资源（Beta），并且从 v1.11 开始默认开启。v1.10 版本使用前需要在kube-apiserver开启--feature-gates=CustomResourceSubresources=true。
1
# resourcedefinition.yaml
2
apiVersion: apiextensions.k8s.io/v1beta1
3
kind: CustomResourceDefinition
4
metadata:
5
name: crontabs.stable.example.com
6
spec:
7
group: stable.example.com
8
version: v1
9
scope: Namespaced
10
names:
11
plural: crontabs
12
singular: crontab
13
kind: CronTab
14
shortNames:
15
- ct
16
# subresources describes the subresources for custom resources.
17
subresources:
18
# status enables the status subresource.
19
status: {}
20
# scale enables the scale subresource.
21
scale:
22
# specReplicasPath defines the JSONPath inside of a custom resource that corresponds to Scale.Spec.Replicas.
23
specReplicasPath: .spec.replicas
24
# statusReplicasPath defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Replicas.
25
statusReplicasPath: .status.replicas
26
# labelSelectorPath defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Selector.
27
labelSelectorPath: .status.labelSelector
Copied!
1
$ kubectl create -f resourcedefinition.yaml
2
$ kubectl create -f- <<EOF
3
apiVersion: "stable.example.com/v1"
4
kind: CronTab
5
metadata:
6
name: my-new-cron-object
7
spec:
8
cronSpec: "* * * * */5"
9
image: my-awesome-cron-image
10
replicas: 3
11
EOF
12
13
$ kubectl scale --replicas=5 crontabs/my-new-cron-object
14
crontabs "my-new-cron-object" scaled
15
16
$ kubectl get crontabs my-new-cron-object -o jsonpath='{.spec.replicas}'
17
5
Copied!
Categories
Categories 用来将 CRD 对象分组，这样就可以使用
kubectl get <category-name> 来查询属于该组的所有对象。1
# resourcedefinition.yaml
2
apiVersion: apiextensions.k8s.io/v1beta1
3
kind: CustomResourceDefinition
4
metadata:
5
name: crontabs.stable.example.com
6
spec:
7
group: stable.example.com
8
version: v1
9
scope: Namespaced
10
names:
11
plural: crontabs
12
singular: crontab
13
kind: CronTab
14
shortNames:
15
- ct
16
# categories is a list of grouped resources the custom resource belongs to.
17
categories:
18
- all
Copied!
1
# my-crontab.yaml
2
apiVersion: "stable.example.com/v1"
3
kind: CronTab
4
metadata:
5
name: my-new-cron-object
6
spec:
7
cronSpec: "* * * * */5"
8
image: my-awesome-cron-image
Copied!
1
$ kubectl create -f resourcedefinition.yaml
2
$ kubectl create -f my-crontab.yaml
3
$ kubectl get all
4
NAME AGE
5
crontabs/my-new-cron-object 3s
Copied!
CRD 控制器
在使用 CRD 扩展 Kubernetes API 时，通常还需要实现一个新建资源的控制器，监听新资源的变化情况，并作进一步的处理。
- 如何注册资源
Foo - 如何创建、删除和查询
Foo对象 - 如何监听
Foo资源对象的变化情况
Kubebuilder
从上面的实例中可以看到从头构建一个 CRD 控制器并不容易，需要对 Kubernetes 的 API 有深入了解，并且RBAC 集成、镜像构建、持续集成和部署等都需要很大工作量。
安装
1
# Install kubebuilder
2
VERSION=1.0.1
3
wget https://github.com/kubernetes-sigs/kubebuilder/releases/download/v${VERSION}/kubebuilder_${VERSION}_linux_amd64.tar.gz
4
tar zxvf kubebuilder_${VERSION}_linux_amd64.tar.gz
5
sudo mv kubebuilder_${VERSION}_linux_amd64 /usr/local/kubebuilder
6
export PATH=$PATH:/usr/local/kubebuilder/bin
7
8
# Install dep kustomize
9
go get -u github.com/golang/dep/cmd/dep
10
go get github.com/kubernetes-sigs/kustomize
Copied!
使用方法
初始化项目
1
mkdir -p $GOPATH/src/demo
2
cd $GOPATH/src/demo
3
kubebuilder init --domain k8s.io --license apache2 --owner "The Kubernetes Authors"
Copied!
创建 API
1
kubebuilder create api --group ships --version v1beta1 --kind Sloop
Copied!
然后按照实际需要修改
pkg/apis/ship/v1beta1/sloop_types.go 和 pkg/controller/sloop/sloop_controller.go 增加业务逻辑。本地运行测试
1
make install
2
make run
Copied!
如果碰到错误ValidationError(CustomResourceDefinition.status): missing required field "storedVersions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1beta1.CustomResourceDefinitionStatus]，可以手动修改config/crds/ships_v1beta1_sloop.yaml:```yaml status: acceptedNames: kind: "" plural: "" conditions: [] storedVersions: []然后运行kubectl apply -f config/crds创建 CRD。
然后就可以用
ships.k8s.io/v1beta1 来创建 Kind 为 Sloop 的资源了，比如1
kubectl apply -f config/samples/ships_v1beta1_sloop.yaml
Copied!
构建镜像并部署控制器
1
# 替换 IMG 为你自己的
2
export IMG=feisky/demo-crd:v1
3
make docker-build
4
make docker-push
5
make deploy
Copied!
kustomize 已经不再支持通配符，因而上述make deploy可能会碰到Load from path ../rbac/*.yaml failed错误，解决方法是手动修改config/default/kustomization.yaml:resources:
kustomize build config/default | kubectl apply -f -部署，默认部署到demo-systemnamespace 中。
文档和测试
1
# run unit tests
2
make test
3
4
# generate docs
5
kubebuilder docs
Copied!
参考文档
最近更新 2yr ago