【k8s-1.34.2安装部署】六.企业级部署cilium-1.18.4网络插件

简介

本章节主要讲解k8s集群网络插件cni的安装，安装的版本是cilium-1.18.4，本文主要讲解对ciium的安装方式，以及cilium各个安装参数的解释。

== 文章导航===

【k8s-1.34.2安装部署】一.系统初始化及k8s集群规划
【k8s-1.34.2安装部署】二.kubernets软件、证书、配置、脚本等文件准备
【k8s-1.34.2安装部署】三.etcd-v3.6.6 TLS版集群安装
【k8s-1.34.2安装部署】四.kubernets master组件kube-apiserver，kube-controller-manager，kube-scheduler安装
【k8s-1.34.2安装部署】五.worker端containerd2.2.1、kubelet-1.34.2安装
【k8s-1.34.2安装部署】六.企业级部署cilium-1.18.4网络插件
【k8s-1.34.2安装部署】七.coredns-v1.13.1安装
【k8s-1.34.2安装部署】八.metric-server-0.8.0安装
【k8s-1.34.2安装部署】九.k8s管理平台xkube安装部署
【k8s-1.34.2安装部署】十.gateway Api v1.4.0安装
【k8s-1.34.2安装部署】十一.metallb-v0.15.2安装

一.安装helm和cilium 客户端命令

cilium的cni通过helm来进行安装，所以首先安装helm，helm的安装在第二章中"安装客户端软件及命令"中有提到，如果没有安装可以在https://github.com/helm/helm/releases/latest 下载二进制文件放到/usr/bin目录下即可。

二.安装cilium

此处的安装是指安装网络插件cni,系统内核要求Kernel >= 6.8,

2.1.安装的命令及参数如下：

1.执行安装命令：注意反斜杠后边不能有空格

helm install cilium cilium/cilium --version 1.18.4  \
  --namespace kube-system \
  --set routingMode=native \
  --set kubeProxyReplacement=true \
  --set autoDirectNodeRoutes=true \
  --set ipv4NativeRoutingCIDR=10.244.0.0/16 \
  --set loadBalancer.mode=hybrid \
  --set loadBalancer.acceleration=native \
  --set k8sServiceHost=apiserver.my-k8s.local \
  --set k8sServicePort=6443 \
  --set bpf.datapathMode=netkit \
  --set bpf.masquerade=true \
  --set bandwidthManager.enabled=true \
  --set bandwidthManager.bbr=true \
  --set ipam.operator.clusterPoolIPv4PodCIDRList=10.244.0.0/16 \
  --set ipam.operator.clusterPoolIPv4MaskSize=24 \
  --set prometheus.enabled=true \
  --set operator.prometheus.enabled=true \
  --set hubble.relay.enabled=true \
  --set hubble.ui.enabled=true \
  --set hubble.metrics.enabled="{dns,drop,tcp,flow,icmp,port-distribution,http}" \
  --set bpf.distributedLRU.enabled=true \
  --set bpf.mapDynamicSizeRatio=0.08 \
  --set ipv4.enabled=true \
  --set enableIPv4BIGTCP=true

2.2.如果提示找不到chat的话，添加helm 安装的 chart地址

helm repo add ali-incubator     https://aliacs-app-catalog.oss-cn-hangzhou.aliyuncs.com/charts-incubator/ 
helm repo add ali-stable    https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
helm repo add cilium https://helm.cilium.io/

2.3.查看安装参数：helm show values cilium/cilium

==参数注释==：

routingMode=native #启用本地路由(Native Routing)，native routing和vxlan模式的区别主要是跨节点通信时，vxlan模式需要封装，native routing模式需要根据目的ip查找路由表转发到其他节点。
autoDirectNodeRoutes=true #表示由cilium在主机上生成Pod的路由规则
loadBalancer.mode=hybrid #hybrid模式时：DSR用于TCP，SNAT用于UDP以避免另外需要的MTU减少。dsr在公有云可能不起作用
loadBalancer.acceleration=native #启用XDP 加速南北向 nodePort 访问
bpf.masquerade=true #IP 地址伪装(Masquerading)切换为基于 eBPF 的模式
kubeProxyReplacement="true" #替换kube-proxy
bandwidthManager.enabled=true #启用带宽管理功能，用于进行带宽限速
bandwidthManager.bbr=true #启用 Pod 的 BBR 网络阻塞控制
prometheus.enabled=true #启用promes监控
hubble.relay.enabled=true #启用hubble.relay来观测流量
hubble.ui.enabled=true #观测流量的ui

helm安装cilium时可以通过此命令查看安装帮助：helm show values cilium/cilium

三.确认cilium的安装状态

1.cilium status

    /¯¯\
 /¯¯\__/¯¯\    Cilium:             OK
 \__/¯¯\__/    Operator:           OK
 /¯¯\__/¯¯\    Envoy DaemonSet:    OK
 \__/¯¯\__/    Hubble Relay:       OK
    \__/       ClusterMesh:        disabled

DaemonSet              cilium                   Desired: 4, Ready: 4/4, Available: 4/4
DaemonSet              cilium-envoy             Desired: 4, Ready: 4/4, Available: 4/4
Deployment             cilium-operator          Desired: 2, Ready: 2/2, Available: 2/2
Deployment             hubble-relay             Desired: 1, Ready: 1/1, Available: 1/1
Deployment             hubble-ui                Desired: 1, Ready: 1/1, Available: 1/1
Containers:            cilium                   Running: 4
                       cilium-envoy             Running: 4
                       cilium-operator          Running: 2
                       clustermesh-apiserver    
                       hubble-relay             Running: 1
                       hubble-ui                Running: 1
Cluster Pods:          23/23 managed by Cilium
Helm chart version:    1.18.4
Image versions         cilium             quay.io/cilium/cilium:v1.18.4@sha256:49d87af187eeeb9e9e3ec2bc6bd372261a0b5cb2d845659463ba7cc10fe9e45f: 4
                       cilium-envoy       quay.io/cilium/cilium-envoy:v1.34.10-1762597008-ff7ae7d623be00078865cff1b0672cc5d9bfc6d5@sha256:1deb6709afcb5523579bf1abbc3255adf9e354565a88c4a9162c8d9cb1d77ab5: 4
                       cilium-operator    quay.io/cilium/operator-generic:v1.18.4@sha256:1b22b9ff28affdf574378a70dade4ef835b00b080c2ee2418530809dd62c3012: 2
                       hubble-relay       quay.io/cilium/hubble-relay:v1.18.4@sha256:6d350cb1c84b847adb152173debef1f774126c69de21a5921a1e6a23b8779723: 1
                       hubble-ui          quay.io/cilium/hubble-ui-backend:v0.13.3@sha256:db1454e45dc39ca41fbf7cad31eec95d99e5b9949c39daaad0fa81ef29d56953: 1
                       hubble-ui          quay.io/cilium/hubble-ui:v0.13.3@sha256:661d5de7050182d495c6497ff0b007a7a1e379648e60830dd68c4d78ae21761d: 1

1. 查看 daemonset 的状态命令：kubectl exec -it -n kube-system ds/cilium -- cilium status --verbose

2. 查看cilium配置：kubectl get cm cilium-config -n kube-system -o yaml
   重启cilium进程命令：kubectl -n kube-system rollout restart ds/cilium

==注意事项:==

1. 此时的coredns还没安装，会报错：level=warning msg="Failed to create peer client for peers synchronization; will try again after the timeout has expired" error="context deadline exceeded" subsys=hubble-relay target="hubble-peer.kube-system.svc.cluster.local:443"
   这是解析错误导致，安装完coredns,然后杀掉hubble-relay 的pod后就正常了
2. 如果镜像无法下载,先将镜像下载到自己的镜像地址,并通过kubectl edit ds cilium -n kube-system 修改配置里的镜像。

cilium所需镜像如下，可以提前下载好:
Image versions         cilium             quay.io/cilium/cilium:v1.18.4@sha256:49d87af187eeeb9e9e3ec2bc6bd372261a0b5cb2d845659463ba7cc10fe9e45f: 3
                       cilium-envoy       quay.io/cilium/cilium-envoy:v1.34.10-1762597008-ff7ae7d623be00078865cff1b0672cc5d9bfc6d5@sha256:1deb6709afcb5523579bf1abbc3255adf9e354565a88c4a9162c8d9cb1d77ab5: 3
                       cilium-operator    quay.io/cilium/operator-generic:v1.18.4@sha256:1b22b9ff28affdf574378a70dade4ef835b00b080c2ee2418530809dd62c3012: 2
                       hubble-relay       quay.io/cilium/hubble-relay:v1.18.4@sha256:6d350cb1c84b847adb152173debef1f774126c69de21a5921a1e6a23b8779723: 1
                       hubble-ui          quay.io/cilium/hubble-ui-backend:v0.13.3@sha256:db1454e45dc39ca41fbf7cad31eec95d99e5b9949c39daaad0fa81ef29d56953: 1
                       hubble-ui          quay.io/cilium/hubble-ui:v0.13.3@sha256:661d5de7050182d495c6497ff0b007a7a1e379648e60830dd68c4d78ae21761d: 1

文档中的cilium相关版本可以在作者本人的阿里云个人进行地址下载：

Image versions         cilium             crpi-44hgz4440mgo9lnt.cn-guangzhou.personal.cr.aliyuncs.com/eeenet/cilium:v1.18.4
                       cilium-envoy       crpi-44hgz4440mgo9lnt.cn-guangzhou.personal.cr.aliyuncs.com/eeenet/cilium-envoy:v1.34.10-1762597008-ff7ae7d623be00078865cff1b0672cc5d9bfc6d5
                       cilium-operator    crpi-44hgz4440mgo9lnt.cn-guangzhou.personal.cr.aliyuncs.com/eeenet/operator-generic:v1.18.4
                       hubble-relay       crpi-44hgz4440mgo9lnt.cn-guangzhou.personal.cr.aliyuncs.com/eeenet/hubble-relay:v1.18.4
                       hubble-ui          crpi-44hgz4440mgo9lnt.cn-guangzhou.personal.cr.aliyuncs.com/eeenet/hubble-ui-backend:v0.13.3
                       hubble-ui          crpi-44hgz4440mgo9lnt.cn-guangzhou.personal.cr.aliyuncs.com/eeenet/hubble-ui:v0.13.3

四.访问cilium的监控页面hubble-ui

将 Hubble UI 改为 NodePort 访问即可
kubectl -n kube-system patch svc hubble-ui -p '{"spec": {"type": "NodePort"}}'