新版 VolumeSnapshot 使用注意事项

基础文档： https://developer.aliyun.com/article/757325

背景： csi-plugin 插件于近期进行一次重大升级，升级到最新版本的 csi-plugin 镜像在使用 VolumeSnapshot 的时候需要做模板上的变更。 使用 csi-plugin:v1.14.8.42-9451f619-aliyun 以及以上版本的镜像需要参照如下内容对模板进行升级

升级方式： 删除原有 csi-snapshotter deployment 按照如下模板部署新的 deployment

csi-snapshotter yaml

kind: Deployment
apiVersion: apps/v1
metadata:
  name: csi-snapshotter
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: csi-snapshotter
  replicas: 1
  template:
    metadata:
      labels:
        app: csi-snapshotter
    spec:
      tolerations:
      - operator: "Exists"
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: node-role.kubernetes.io/master
                operator: Exists
      priorityClassName: system-node-critical
      serviceAccount: admin
      hostNetwork: true
      containers:
        - name: snapshot-controller
          image: registry.cn-hangzhou.aliyuncs.com/plugins/snapshot-controller:v2.0.1
          args:
            - "--v=5"
            - "--leader-election=false"
          imagePullPolicy: Always
        - name: disk-snapshotter
          image: registry.cn-hangzhou.aliyuncs.com/plugins/csi-snapshotter:v2.1.1
          args:
            - "--csi-address=$(ADDRESS)"
          env:
            - name: ADDRESS
              value: /var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/csi.sock
          imagePullPolicy: "Always"
          volumeMounts:
            - name: disk-provisioner-dir
              mountPath: /var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/
        - name: csi-diskprovisioner
          securityContext:
            privileged: true
            capabilities:
              add: ["SYS_ADMIN"]
            allowPrivilegeEscalation: true
          image: registry.cn-hangzhou.aliyuncs.com/acs/csi-plugin:v1.14.8.42-9451f619-aliyun
          imagePullPolicy: "Always"
          args:
            - "--endpoint=$(CSI_ENDPOINT)"
            - "--v=2"
            - "--driver=disk"
          env:
            - name: CSI_ENDPOINT
              value: unix://var/lib/kubelet/csi-plugins/driverplugin.csi.alibabacloud.com-replace/csi.sock
            - name: SERVICE_PORT
              value: "11271"
            - name: SERVICE_TYPE
              value: "provisioner"
          volumeMounts:
            - name: host-log
              mountPath: /var/log/
            - name: disk-provisioner-dir
              mountPath: /var/lib/kubelet/csi-plugins/diskplugin.csi.alibabacloud.com/
            - name: etc
              mountPath: /host/etc
      volumes:
        - name: disk-provisioner-dir
          emptyDir: {}
        - name: host-log
          hostPath:
            path: /var/log/
        - name: etc
          hostPath:
            path: /etc

注意点

1. yaml 中 driverplugin.csi.alibabacloud.com-replace 会被替换， 不用理会
2. yaml 中 --driver=disk 同样会被替换， VolumeSnapshotClass driver 保留原先 diskplugin.csi.alibabacloud.com 即可

升级方式： 删除原有 csi-provisioner deployment 按照如下模板部署新的 deployment

csi-provisioner.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: csi-provisioner
  name: csi-provisioner
  namespace: kube-system
spec:
  progressDeadlineSeconds: 600
  replicas: 2
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: csi-provisioner
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: csi-provisioner
    spec:
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - preference:
              matchExpressions:
              - key: node-role.kubernetes.io/master
                operator: Exists
            weight: 1
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: type
                operator: NotIn
                values:
                - virtual-kubelet
      containers:
      - args:
        - --provisioner=diskplugin.csi.alibabacloud.com
        - --csi-address=$(ADDRESS)
        - --feature-gates=Topology=True
        - --volume-name-prefix=disk
        - --strict-topology=true
        - --timeout=150s
        - --enable-leader-election=true
        - --leader-election-type=leases
        - --retry-interval-start=500ms
        - --v=5
        env:
        - name: ADDRESS
          value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
        image: registry.cn-hangzhou.aliyuncs.com/acs/csi-provisioner:v1.6.0-fc9e11563-ack
        imagePullPolicy: Always
        name: external-disk-provisioner
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
          name: disk-provisioner-dir
      - args:
        - --v=5
        - --csi-address=$(ADDRESS)
        - --leader-election=true
        env:
        - name: ADDRESS
          value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
        image: registry-vpc.cn-beijing.aliyuncs.com/acs/csi-attacher:v2.1.0
        imagePullPolicy: Always
        name: external-disk-attacher
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
          name: disk-provisioner-dir
      - args:
        - --v=5
        - --csi-address=$(ADDRESS)
        - --leader-election
        env:
        - name: ADDRESS
          value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
        image: registry-vpc.cn-beijing.aliyuncs.com/acs/csi-resizer:v0.3.0
        imagePullPolicy: Always
        name: external-disk-resizer
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
          name: disk-provisioner-dir
      - args:
        - --provisioner=nasplugin.csi.alibabacloud.com
        - --csi-address=$(ADDRESS)
        - --volume-name-prefix=nas
        - --timeout=150s
        - --enable-leader-election=true
        - --leader-election-type=leases
        - --retry-interval-start=500ms
        - --v=5
        env:
        - name: ADDRESS
          value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
        image: registry-vpc.cn-beijing.aliyuncs.com/acs/csi-provisioner:v1.4.0-aliyun
        imagePullPolicy: Always
        name: external-nas-provisioner
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
          name: nas-provisioner-dir
      - args:
        - --endpoint=$(CSI_ENDPOINT)
        - --v=2
        - --driver=nas,disk
        env:
        - name: CSI_ENDPOINT
          value: unix://var/lib/kubelet/csi-provisioner/driverplugin.csi.alibabacloud.com-replace/csi.sock
        - name: MAX_VOLUMES_PERNODE
          value: "15"
        - name: SERVICE_TYPE
          value: provisioner
        image: registry.cn-hangzhou.aliyuncs.com/acs/csi-plugin:v1.14.8.42-9451f619-aliyun
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 5
          httpGet:
            path: /healthz
            port: healthz
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 30
          successThreshold: 1
          timeoutSeconds: 3
        name: csi-provisioner
        ports:
        - containerPort: 11270
          hostPort: 11270
          name: healthz
          protocol: TCP
        resources:
          limits:
            cpu: 100m
            memory: 100Mi
          requests:
            cpu: 100m
            memory: 100Mi
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            add:
            - SYS_ADMIN
          privileged: true
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /dev
          mountPropagation: HostToContainer
          name: host-dev
        - mountPath: /var/log/
          name: host-log
        - mountPath: /host/etc
          name: etc
        - mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
          name: disk-provisioner-dir
        - mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
          name: nas-provisioner-dir
      dnsPolicy: ClusterFirst
      hostNetwork: true
      priorityClassName: system-node-critical
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: admin
      serviceAccountName: admin
      terminationGracePeriodSeconds: 30
      volumes:
      - emptyDir: {}
        name: disk-provisioner-dir
      - emptyDir: {}
        name: nas-provisioner-dir
      - hostPath:
          path: /var/log/
          type: ""
        name: host-log
      - hostPath:
          path: /dev
          type: ""
        name: host-dev
      - hostPath:
          path: /etc
          type: ""
        name: etc

注意点

1. external-disk-provisioner 的版本需要保持在1.6 之上， 否则不会识别 pvc 中的 datasource 字段 导致 restore 失败

权限

1. 保证 ACK worker role 拥有 ecs 快照相关操作权限。 下面列出所需权限列表

• CreateSnapshot
• DescribeSnapshotAttribute
• DescribeSnapshots
• DescribeSnapshotLinks
• CreateAutoSnapshotPolicy（option）
• ApplyAutoSnapshotPolicy（option）
• CancelAutoSnapshotPolicy（option）
• DeleteAutoSnapshotPolicy（option）