Kubespray

Kubespray is a project under the Kubernetes incubator. Its mission is to provide a production-ready Kubernetes deployment solution. The project is based on Ansible Playbook to define system and Kubernetes cluster deployment tasks, with the following characteristics:

It can be deployed on AWS, GCE, Azure, OpenStack, and bare metal.
It allows for the deployment of highly available Kubernetes clusters.
It is composable, allowing users to choose Network Plugin (flannel, calico, canal, weave) for deployment.
It supports various Linux distributions (CoreOS, Debian Jessie, Ubuntu 16.04, CentOS/RHEL7).

This article will explain how to deploy Kubernetes to bare metal nodes using Kubespray. The versions will be as follows:

Kubernetes v1.7.3
Etcd v3.2.4
Flannel v0.8.0
Docker v17.04.0-ce

Node Information

The operating system for the installation test environment will be Ubuntu 16.04 Server and the other details are as follows:

IP Address	Role	CPU	Memory
192.168.121.179	master1 + deploy	2	4G
192.168.121.106	node1	2	4G
192.168.121.197	node2	2	4G
192.168.121.123	node3	2	4G

Here, the master is the primary control node, and the node is the work node.

Preparatory Information

All nodes' networks can communicate with each other.
The deployment node (here, master1) can log in to other nodes without needing SSH passwords.
All nodes possess Sudoer permissions and don't require password input.
All nodes need to have Python installed.
All nodes need to set /etc/hosts to resolve all hosts.
Modify all nodes' /etc/resolv.conf

$ echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf

The deployment node (here, master1) installs Ansible >= 2.3.0.

The process for installing Ansible on Ubuntu 16.04 is as follows:

$ sudo sed -i 's/us.archive.ubuntu.com/tw.archive.ubuntu.com/g' /etc/apt/sources.list
$ sudo apt-get install -y software-properties-common
$ sudo apt-add-repository -y ppa:ansible/ansible
$ sudo apt-get update && sudo apt-get install -y ansible git cowsay python-pip python-netaddr libssl-dev

Installing Kubespray and Preparing Deployment Information

First, install kubespray-cli through pypi. Although the official sources say they have switched to a Go language version of the tool, it hasn't been updated, so we'll use the pypi version for now:

$ sudo pip install -U kubespray

After installation, add a configuration file ~/.kubespray.yml and include the following content:

$ cat <<EOF> ~/.kubespray.yml
kubespray_git_repo: "https://github.com/kubernetes-incubator/kubespray.git"
# Logging options
loglevel: "info"
EOF

Then use the kubespray cli to generate an inventory file:

$ kubespray prepare --masters master1 --etcds master1 --nodes node1 node2 node3

Add some content in the inventory.cfg:

$ vim ~/.kubespray/inventory/inventory.cfg

[all]
master1  ansible_host=192.168.121.179   ansible_user=root ip=192.168.121.179
node1    ansible_host=192.168.121.106 ansible_user=root ip=192.168.121.106
node2    ansible_host=192.168.121.197 ansible_user=root ip=192.168.121.197
node3    ansible_host=192.168.121.123 ansible_user=root ip=192.168.121.123

[kube-master]
master1

[kube-node]
node1
node2
node3

[etcd]
master1

[k8s-cluster:children]
kube-node
kube-master

You can also create a new inventory to describe the deployment nodes.

After completing the above, execute the following command to deploy the Kubernetes cluster:

$ time kubespray deploy --verbose -u root -k .ssh/id_rsa -n flannel
Run kubernetes cluster deployment with the above command ? [Y/n]y
...
master1                    : ok=368  changed=89   unreachable=0    failed=0
node1                      : ok=305  changed=73   unreachable=0    failed=0
node2                      : ok=276  changed=62   unreachable=0    failed=0
node3                      : ok=276  changed=62   unreachable=0    failed=0

Kubernetes deployed successfully

The -n refers to the type of network plugin to be deployed, currently supporting calico, flannel, weave, and canal.

Verifying the Cluster

After Ansible has run, if no errors have occurred, you can start operating the Kubernetes, such as obtaining version information:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.7.3+coreos.0", GitCommit:"9212f77ed8c169a0afa02e58dce87913c6387b3e", GitTreeState:"clean", BuildDate:"2017-04-04T00:32:53Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.7.3+coreos.0", GitCommit:"9212f77ed8c169a0afa02e58dce87913c6387b3e", GitTreeState:"clean", BuildDate:"2017-04-04T00:32:53Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

Get the current cluster node status:

$ kubectl get node
NAME      STATUS                     AGE       VERSION
master1   Ready,SchedulingDisabled   11m       v1.7.3+coreos.0
node1     Ready                      11m       v1.7.3+coreos.0
node2     Ready                      11m       v1.7.3+coreos.0
node3     Ready                      11m       v1.7.3+coreos.

Check the current cluster Pod status:

$ kubectl get po -n kube-system
NAME                                  READY     STATUS    RESTARTS   AGE
dnsmasq-975202658-6jj3n               1/1       Running   0          14m
dnsmasq-975202658-h4rn9               1/1       Running   0          14m
dnsmasq-autoscaler-2349860636-kfpx0   1/1       Running   0          14m
flannel-master1                       1/1       Running   1          14m
flannel-node1                         1/1       Running   1          14m
flannel-node2                         1/1       Running   1          14m
flannel-node3                         1/1       Running   1          14m
kube-apiserver-master1                1/1       Running   0          15m
kube-controller-manager-master1       1/1       Running   0          15m
kube-proxy-master1                    1/1       Running   1          14m
kube-proxy-node1                      1/1       Running   1          14m
kube-proxy-node2                      1/1       Running   1          14m
kube-proxy-node3                      1/1       Running   1          14m
kube-scheduler-master1                1/1       Running   0          15m
kubedns-1519522227-thmrh              3/3       Running   0          14m
kubedns-autoscaler-2999057513-tx14j   1/1       Running   0          14m
nginx-proxy-node1                     1/1       Running   1          14m
nginx-proxy-node2                     1/1       Running   1          14m
nginx-proxy-node3                     1/1       Running   1          14m

最后更新于11个月前