kubernetes 1.8 高可用安装（四）

4、安装kubernetes node

Kubernetes的一个Node节点上需要运行如下组件：

• Docker，目前安装的是docker-1.12.6

• kubelet

• kube-proxy 使用daemonset安装

4.1 安装kubelet和cni

安装rpm包

yum localinstall -y kubelet-1.8.0-1.x86_64.rpm kubernetes-cni-0.5.1-1.x86_64.rpm

在任一master节点创建ClusterRoleBinding

kubectl create clusterrolebinding kubelet-bootstrap \
  --clusterrole=system:node-bootstrapper \
  --user=kubelet-bootstrap

4.2将证书和配置文件同步到本机

rsync -avSH rsync://master_ip/k8s/pki /etc/kubernetes/
rsync -avSH rsync://master_ip/k8s/bootstrap.kubeconfig /etc/kubernetes/

4.3 配置kubelet

/etc/systemd/system/kubelet.service.d/kubelet.conf

[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_SYSTEM_PODS_ARGS=--pod-manifest-path=/etc/kubernetes/manifests --allow-privileged=true"
Environment="KUBELET_NETWORK_ARGS=--network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin"
Environment="KUBELET_DNS_ARGS=--cluster-dns=10.96.0.12 --cluster-domain=cluster.local"
Environment="KUBELET_AUTHZ_ARGS=--authorization-mode=Webhook --client-ca-file=/etc/kubernetes/pki/ca.pem"
Environment="KUBELET_CADVISOR_ARGS=--cadvisor-port=0"
Environment="KUBELET_CGROUP_ARGS=--cgroup-driver=cgroupfs"
Environment="KUBELET_CERTIFICATE_ARGS=--rotate-certificates=true --cert-dir=/var/lib/kubelet/pki"
Environment="KUBELET_EXTRA_ARGS=--v=2 --pod-infra-container-image=foxchan/pause-amd64:3.0 --fail-swap-on=false"
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ_ARGS $KUBELET_CADVISOR_ARGS $K
UBELET_CGROUP_ARGS $KUBELET_CERTIFICATE_ARGS $KUBELET_EXTRA_ARGS

4.4 配置kube-proxy

修改后启动kubelet

systemctl daemon-reload
systemctl start kubelet

由于采用了 TLS Bootstrapping，所以 kubelet 启动后不会立即加入集群，而是进行证书申请，
看日志

Oct 24 16:45:43  kubelet[240975]: I1024 16:45:43.566069  240975 bootstrap.go:57] Using bootstrap kubeconfig to generate TLS client cert, key and kubeconfig file

看csr,仍然是pending状态

[root@kvm-master manifests]# kubectl get csr
NAME                                                   AGE       REQUESTOR           CONDITION
node-csr-VJFRWBpJqhe3lpLKPULmJ9wfYeF0xoMQF8VzfcvYyqw   2h        kubelet-bootstrap   Approved,Issued
node-csr-yCn3MIUz-luhqwEVva1haugCmoz48ykxU7x4er3pfQs   44s       kubelet-bootstrap   Pending

需要在 master 允许其证书申请

kubectl get csr | grep Pending | awk '{print $1}' | xargs kubectl certificate approve

此时看node已经加入集群

[root@kvm-master manifests]# kubectl get nodes
NAME            STATUS     ROLES     AGE       VERSION
node2   NotReady   <none>    5m        v1.8.0
node1    Ready      <none>    1h        v1.8.0

因为kubelet配置了network-plugin=cni，但是还没安装，所以状态会是NotReady,不想看这个报错或者不需要网络，就可以修改kubelet配置文件，去掉network-plugin=cni 就可以了。

Oct 25 15:48:15 localhost kubelet: W1025 15:48:15.584765  240975 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Oct 25 15:48:15 localhost kubelet: E1025 15:48:15.585057  240975 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized

创建kube-proxy 相关文件

在master操作
kubectl apply -f kube-proxy-rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-proxy
  namespace: kube-system
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: system:kube-proxy
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
subjects:
  - kind: ServiceAccount
    name: kube-proxy
    namespace: kube-system
roleRef:
  kind: ClusterRole
  name: system:node-proxier
  apiGroup: rbac.authorization.k8s.io

kubectl apply -f kubeproxy-ds.yaml

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  labels:
    k8s-app: kube-proxy
  name: kube-proxy
  namespace: kube-system
spec:
  selector:
    matchLabels:
      k8s-app: kube-proxy
  template:
    metadata:
      labels:
        k8s-app: kube-proxy
    spec:
      containers:
      - command:
        - /bin/sh
        - -c
        - /usr/local/bin/kube-proxy
          --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig
          --cluster-cidr=10.96.0.0/12
          --conntrack-max-per-core=655360
          --conntrack-min=655360
          --conntrack-tcp-timeout-established=1h
          --conntrack-tcp-timeout-close-wait=60s
          --v=2 1>>/var/log/kube-proxy.log 2>&1
        name: kube-proxy
        image: foxchan/kube-proxy-amd64:v1.8.1
        imagePullPolicy: IfNotPresent
        securityContext:
          privileged: true
        volumeMounts:
        - mountPath: /etc/kubernetes/
          name: k8s
        - mountPath: /var/log/kube-proxy.log
          name: logfile
        - mountPath: /run/xtables.lock
          name: xtables-lock
        - mountPath: /lib/modules
          name: modprobe
      hostNetwork: true
      serviceAccountName: kube-proxy
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
      volumes:
      - hostPath:
          path: /etc/kubernetes
        name: k8s
      - hostPath:
          path: /var/log/kube-proxy.log
        name: logfile
      - hostPath:
          path: /run/xtables.lock
          type: FileOrCreate
        name: xtables-lock
      - hostPath:
          path: /lib/modules
          type: ""
        name: modprobe
  updateStrategy:
    rollingUpdate:
      maxUnavailable: 1
    type: RollingUpdate

查看 proxy 是否正常

[root@kvm-master kubeproxy]# kubectl get pods -n kube-system
NAME               READY     STATUS    RESTARTS   AGE
kube-proxy-rw2bt   1/1       Running   0          1m
kube-proxy-sct84   1/1       Running   0          1m