Kubernetes是用于自动部署、扩展和管理“容器化应用程序”的开源系统。该系统由Google设计并捐赠给Cloud Native Computing Foundation来使用。 它旨在提供“跨主机集群的自动部署、扩展以及运行应用程序容器的平台”。 它支持一系列容器工具, 包括Docker等。

环境介绍

基本环境

  • CentOS Linux release 7.5.1804 (Core)
  • JDK1.8.0_161
  • Kubernetes v1.5.2
  • yum源：清华大学

部署规划

Master:

  • ip: 10.10.202.158
  • hostname: apm-slave-02
  • 安装节点
    • docker
    • etcd
    • flannel
    • kube-apiserver
    • kube-scheduler
    • kube-controller-manager

Node:

  • ip: 10.10.202.159
  • hostname: apm-slave-03
  • 安装节点
    • docker
    • flannel
    • kubelet
    • kube-proxy

防火墙

1
2
systemctl disable firewalld.service
systemctl stop firewalld.service

部署Master节点

安装Docker

1
yum install docker

启动docker，并加入开机启动

1
2
systemctl start docker
systemctl enable docker

安装etcd

1
yum install etcd -y

配置etcd，编辑 /etc/ectd/etcd.conf

1
vim /etc/etcd/etcd.conf 
1
2
3
4
5
6
- ETCD_LISTEN_CLIENT_URLS="http://127.0.0.1:2379"
+ ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379,http://0.0.0.0:4001"
- ETCD_NAME="default"
+ ETCD_NAME="master"
- ETCD_ADVERTISE_CLIENT_URLS="http://127.0.0.1:2379"
+ ETCD_ADVERTISE_CLIENT_URLS="http://apm-slave02:2379,http://apm-slave02:4001"

启动ectd

1
systemctl start etcd

查看服务是否启动

1
2
3
systemctl is-active etcd

active

获取etcd的健康指标

1
etcdctl -C http://apm-slave02:2379 cluster-health
1
2
member 8e9e05c52164694d is healthy: got healthy result from http://apm-slave02:2379
cluster is healthy

加入开机启动

1
systemctl enable etcd

安装kubernetes

1
yum install kubernetes

配置kubernetes，编辑 /etc/kubernetes/下面的apiserver、config、scheduler配置文件

apiserver

1
vim /etc/kubernetes/apiserver
1
2
3
4
5
6
- KUBE_API_ADDRESS="--address=127.0.0.1"
+ KUBE_API_ADDRESS="--address=0.0.0.0"
- KUBE_ETCD_SERVERS="--etcd-servers=http://localhost:2379"
+ KUBE_ETCD_SERVERS="--etcd-servers=http://10.10.202.158:2379"
- KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,ServiceAccount,SecurityContextDeny,ResourceQuota"
+ KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"

config

1
vim /etc/kubernetes/config
1
2
- KUBE_MASTER="--master=http://127.0.0.1:8080"
+ KUBE_MASTER="--master=http://10.10.202.158:8080"

启动Master组件

1
2
3
systemctl start kube-apiserver.service
systemctl start kube-controller-manager.service
systemctl start kube-scheduler.service

加入开机启动

1
2
3
systemctl enable kube-apiserver.service
systemctl enable kube-controller-manager.service
systemctl enable kube-scheduler.service

检查

1
2
3
4
5
6
 systemctl list-unit-files |grep kube
kube-apiserver.service                        enabled 
kube-controller-manager.service               enabled 
kube-proxy.service                            disabled
kube-scheduler.service                        enabled 
kubelet.service                               disabled

安装flannel

1
yum install flannel

配置flannel

1
vim /etc/sysconfig/flanneld
1
2
-FLANNEL_ETCD_ENDPOINTS="http://127.0.0.1:2379"
+FLANNEL_ETCD_ENDPOINTS="http://10.10.202.158:2379"

配置etcd中flannel的key

1
etcdctl mk /atomic.io/network/config '{ "Network": "10.0.0.0/16" }'

启动flannel

1
systemctl start flanneld.service

设置开机启动

1
systemctl enable flanneld.service

检查服务：

1
2
3
4
5
6
7
systemctl is-active  kube-apiserver.service kube-controller-manager.service kube-scheduler.service etcd flanneld.service

active
active
active
active
active

注意启动顺序 etcd —> kubernetes

部署Node节点

安装Docker

1
yum install docker

启动docker，并加入开机启动

1
2
systemctl start docker
systemctl enable docker

安装flannel

1
yum install flannel

配置flannel

1
vim /etc/sysconfig/flanneld
1
2
-FLANNEL_ETCD_ENDPOINTS="http://127.0.0.1:2379"
+FLANNEL_ETCD_ENDPOINTS="http://10.10.202.158:2379"

配置etcd中flannel的key

1
etcdctl mk /atomic.io/network/config '{ "Network": "10.0.0.0/16" }'

启动flannel

1
systemctl start flanneld.service

设置开机启动

1
systemctl enable flanneld.service

安装kubernetes

1
yum install kubernetes

node节点需要运行如下组件：

  • kubelet
  • kubernets-proxy

编辑/etc/kubernetes/config

1
2
-KUBE_MASTER="--master=http://127.0.0.1:8080"
+KUBE_MASTER="--master=http://10.10.202.158:8080"

编辑/etc/kubernetes/kubelet

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
###
# kubernetes kubelet (minion) config

# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"

# The port for the info server to serve on
# KUBELET_PORT="--port=10250"

# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname-override=apm-slave03"

# location of the api-server
KUBELET_API_SERVER="--api-servers=http://10.10.202.158:8080"

# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"

# Add your own!
KUBELET_ARGS=""

注意此处的KUBELET_POD_INFRA_CONTAINER，它的值是registry.access.redhat.com/rhel7/pod-infrastructure:latest，如果没有安装rhsm会出错，具体请见《解决k8s出现pod服务一直处于ContainerCreating状态的问题》 。

启动kubernetes服务

1
2
systemctl start kubelet.service
systemctl start kube-proxy.service

加入开机启动

1
2
systemctl enable kubelet.service
systemctl enable kube-proxy.service

检测Node节点的服务

1
2
3
4
5
systemctl is-active kube-proxy.service kubelet.service flanneld.service

active
active
active

在Master（10.10.202.158）上执行如下命令：

1
2
3
kubectl get endpoints
NAME         ENDPOINTS            AGE
kubernetes   10.10.202.158:6443   2d
1
2
3
kubectl get nodes
NAME          STATUS    AGE
apm-slave03   Ready     22h

此时，k8s集群就已经安装完成。

安装Kubernetes-Dashboard

在Master节点的机器上创建两个文件dashboard-controller.yaml和dashboard-service.yaml，其具体内容为：

dashboard-controller.yaml:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: kubernetes-dashboard
  namespace: kube-system
  labels:
    k8s-app: kubernetes-dashboard
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
spec:
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: ''
    spec:
      serviceAccountName: dashboard
      containers:
      - name: kubernetes-dashboard
        image: registry.cn-hangzhou.aliyuncs.com/google-containers/kubernetes-dashboard-amd64:v1.4.2
        resources:
          limits:
            cpu: 100m
            memory: 50Mi
          requests:
            cpu: 100m
            memory: 50Mi
        ports:
        - containerPort: 9090
        args:
        - --apiserver-host=http://10.10.202.158:8080
        livenessProbe:
          httpGet:
            path: /
            port: 9090
          initialDelaySeconds: 30
          timeoutSeconds: 30

注意- --apiserver-host=http://10.10.202.158:8080，请修改成自己的地址。
dashboard-service.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
apiVersion: v1
kind: Service
metadata:
  name: kubernetes-dashboard
  namespace: kube-system
  labels:
    k8s-app: kubernetes-dashboard
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
spec:
  type: NodePort
  selector:
    k8s-app: kubernetes-dashboard
  ports:
  - port: 80
    targetPort: 9090

执行如下命令

1
kubectl create -f .
1
2
deployment "kubernetes-dashboard" created
service "kubernetes-dashboard" created
1
2
3
kubectl get deployments --all-namespaces                             
NAMESPACE     NAME                   DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
kube-system   kubernetes-dashboard   1         1         1            1           5s

看到可用节点为1个。

1
2
3
kubectl get pods -n kube-system
NAME                                    READY     STATUS    RESTARTS   AGE
kubernetes-dashboard-2620295069-12qfj   1/1       Running   0          3h

访问http://10.10.202.158:8080/ui/即可打开Dashboard