一、亲和性调度简介
1.1 亲和性简介
亲和性调度是指通过配置的形式,实现优先选择满足条件的Node进行调度,如果没有,也可以调度到不满足条件的节点上,使调度更加灵活
亲和性(Affinity)主要分为三类:
- 节点亲和性(nodeAffinity):以node为目标,解决pod可以调度到哪些node的问题
- pod亲和性(podAffinity):以pod为目标,解决pod可以和哪些已经存在pod部署到同一个拓扑域中的问题
- pod反亲和性(podAntiAffinity):以pod为目标,解决pod不能和哪些已存在的pod部署在统一个拓扑域中的问题
1.2 亲和性和反亲和性说明
- 亲和性:如果两个应用频繁交互,那就有必要利用亲和性让两个应用尽可能靠近,这样可以减少因网络通信而带来的性能损耗
- 反亲和性:当应用采用多副本部署时,有必要采用反亲和性让各个应用实例分布在各个node节点上,这样可以提高服务的高可用性
二、亲和性配置
2.1 节点亲和性(NodeAffinity)
2.1.1 NodeAffinity配置项说明
pod.spec.affinity.nodeAffinity
requiredDuringSchedulingIgnoredDuringExecution Node节点必须满足指定的所有规则才可以,相当于硬限制
nodeSelectorTerms 节点选择列表
matchFields 按节点字段列出的节点选择器要求列表
matchExpressions 按节点标签列出的节点选择器要求列表(推荐)
key 键
values 值
operator 关系符,支持Exists,DoesNotExist,In,NotIn,Gt,Lt
preferredDuringSchedulingIgnoredDuringExecution 优先调整到满足指定的规则的Node,相当于软限制(倾向)
preference 一个节点选择器,与相应的权重相关联
matchFields 按节点字段列出的节点选择器要求列表
matchExpressions 按节点标签列出的节点选择器要求列表(推荐)
key 键
values 值
operator 关系符,支持In,NotIn,Exists,DoesNotExist,Gt,Lt
weight 倾向权重,范围1-100
关系运算符说明:
- matchExpressions:
- key: nodeenv # 匹配存在标签的key为nodeenv的节点
operator: Exists
- key: nodeenv # 匹配标签的key为nodeenv,且value是 "xxx" 或 "yyy" 的节点
operator: In
values: ["xxx","yyy"]
- key: nodeenv # 匹配标签的key为nodeenv,且value大于 "xxx" 的节点
operator: Gt
values: "xxx"
2.1.2 使用required类型的节点亲和性
首先使用如下命令先给node1和node2打标签
[root@master resource_manage]# kubectl label nodes node1 nodeenv=test
node/node1 labeled
[root@master resource_manage]# kubectl label nodes node2 nodeenv=demo
node/node2 labeled
[root@master resource_manage]# kubectl get node --show-labels
NAME STATUS ROLES AGE VERSION LABELS
master Ready control-plane,master 10d v1.21.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=master,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers=
node1 Ready <none> 10d v1.21.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=node1,kubernetes.io/os=linux,nodeenv=test
node2 Ready <none> 10d v1.21.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=node2,kubernetes.io/os=linux,nodeenv=demo
[root@master resource_manage]#
编辑pod_nodeaffinity_required.yaml文件,内容如下:
apiVersion: v1
kind: Namespace
metadata:
name: dev
---
apiVersion: v1
kind: Pod
metadata:
name: pod-nginx
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: nodeenv
operator: In
values: ["demo","demo1"]
使用如下命令创建资源
[root@master resource_manage]# kubectl apply -f pod_nodeaffinity_required.yaml
namespace/dev unchanged
pod/pod-nginx created
[root@master resource_manage]#
如下,可以查询到被调度到node2节点上,符合设置调度规则
[root@master resource_manage]# kubectl get pod -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-nginx 1/1 Running 0 118s 10.244.2.45 node2 <none> <none>
[root@master resource_manage]#
如下,删除资源
[root@master resource_manage]# kubectl delete -f pod_nodeaffinity_required.yaml
namespace "dev" deleted
pod "pod-nginx" deleted
[root@master resource_manage]#
2.1.3 使用prefered类型的节点亲和性
编辑pod_nodeaffinity_preferred.yaml文件,内容如下:
apiVersion: v1
kind: Namespace
metadata:
name: dev
---
apiVersion: v1
kind: Pod
metadata:
name: pod-nginx
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 10
preference:
matchExpressions:
- key: nodeenv
operator: In
values: ["demo2","demo1"]
使用如下命令创建资源
[root@master resource_manage]# kubectl apply -f pod_nodeaffinity_preferred.yaml
namespace/dev unchanged
pod/pod-nginx created
[root@master resource_manage]#
查看结果如下,虽然这里不符合调度条件,但仍然能调度到node2节点,因为这里的调度不是强制性的
[root@master resource_manage]# kubectl get pod -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-nginx 1/1 Running 0 57m 10.244.2.46 node2 <none> <none>
[root@master resource_manage]#
使用如下命令删除资源
[root@master resource_manage]# kubectl delete -f pod_nodeaffinity_preferred.yaml
namespace "dev" deleted
pod "pod-nginx" deleted
[root@master resource_manage]#
2.1.4 Node亲和性注意事项
- 1 如果同时定义了nodeSelector和NodeAffinity,name必须两个条件都得到满足,Pod才能运行在指定的Node上
- 2 如果NodeAffinity指定了多个nodeSelectorTerms,name只需要其中一个能匹配成功即可
- 3 如果一个nodeSelectorTerms中有多个matchExpressions,则一个节点必须满足左右的才能匹配成功
- 4 如果一个pod所在的Node在Pod运行期间其标签发生了改变,不再符合该Pod的节点亲和性需求,则系统忽略此变化
2.2 Pod亲和性(PodAffinity)
2.2.1 Pod亲和性配置项
pod.spec.affinity.podAffinity
requiredDuringSchedulingIgnoredDuringExecution 硬限制
namespace 指定参照pod的namespace
topologyKey 指定调度作用域
labelSelector 标签选择器
matchExpressions 按节点标签列出的节点选择器要求列出(推荐)
key 键
values 值
operator 关系符,支持 In,NotIn,Exists,NoesNotExist
matchLabels 指多个matchExpressions映射的内容
preferredDuringSchedulingIgnoredDuringExecution 软限制
PodAffinityTerm 选项
namespace
topologyKey
labelSelector
matchExpressions:
key 键
values 值
operator
matchLabels
其中topologyKey用于指定调度的作用域,如:
- 如果指定为Kubernetes.io/hostname,那就是以Node节点为分区范围
- 如果指定为beta.kubernets.io/os,则以node节点的操作系统类型来区分
2.2.2 Pod亲和性实例演示
首先使用如下pod_podaffinnity_target.yaml文件创建一个dev命名空间以及一个目标Pod
apiVersion: v1
kind: Namespace
metadata:
name: dev
---
apiVersion: v1
kind: Pod
metadata:
name: pod-podaffinity-target
namespace: dev
labels:
podenv: pro
spec:
containers:
- name: nginx
image: nginx:1.17.1
nodeName: node1
使用如下命令创建资源
[root@master resource_manage]# kubectl apply -f pod_podaffinnity_target.yaml
namespace/dev created
pod/pod-podaffinity-target created
[root@master resource_manage]#
然后编写pod亲和性的yaml文件pod_podaffinity_required.yaml,内容如下:
apiVersion: v1
kind: Pod
metadata:
name: pod-podaffinity-required
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity:
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: podenv
operator: In
values: ["pro"]
topologyKey: kubernetes.io/hostname
使用如下命令创建资源
[root@master resource_manage]# kubectl apply -f pod_podaffinity_required.yaml
pod/pod-podaffinity-required created
[root@master resource_manage]#
通过如下命令可以查看到pod亲和性生效了,此两个pod在一个节点上
[root@master resource_manage]# kubectl get pod -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-podaffinity-required 1/1 Running 0 2m7s 10.244.1.21 node1 <none> <none>
pod-podaffinity-target 1/1 Running 0 13m 10.244.1.20 node1 <none> <none>
[root@master resource_manage]#
2.3 Pod反亲和性(PodAntiAffinity)
Pod 反亲和性主要是设置新建的pod与已知pod不在同一个节点上,配置内容和pod亲和性几乎完全一样,这里直接以实例演示
首先使用如下pod_podaffinnity_target.yaml文件创建一个dev命名空间以及一个目标Pod
apiVersion: v1
kind: Namespace
metadata:
name: dev
---
apiVersion: v1
kind: Pod
metadata:
name: pod-podaffinity-target
namespace: dev
labels:
podenv: pro
spec:
containers:
- name: nginx
image: nginx:1.17.1
nodeName: node1
使用如下命令创建资源
[root@master resource_manage]# kubectl apply -f pod_podaffinnity_target.yaml
namespace/dev created
pod/pod-podaffinity-target created
[root@master resource_manage]#
然后编辑pod反亲和性yaml文件pod_podantiaffinity_required.yaml,内容如下:
apiVersion: v1
kind: Pod
metadata:
name: pod-podaffinity-required
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: podenv
operator: In
values: ["pro"]
topologyKey: kubernetes.io/hostname
然后使用如下命令创建
[root@master resource_manage]# kubectl apply -f pod_podantiaffinity_required.yaml
pod/pod-podaffinity-required created
[root@master resource_manage]#
通过如下命令可以看出此时两个pod已经分别在两个节点上了
[root@master resource_manage]# kubectl get pod -n dev -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod-podaffinity-required 1/1 Running 0 79s 10.244.2.47 node2 <none> <none>
pod-podaffinity-target 1/1 Running 0 2m42s 10.244.1.22 node1 <none> <none>
[root@master resource_manage]#