etcdctl的v3版本与v2版本使用命令有所不同，本文介绍etcdctl v3版本的命令工具的使用方式。
1. etcdctl的安装
etcdctl的二进制文件可以在 github.com/coreos/etcd/releases 选择对应的版本下载，例如可以执行以下install_etcdctl.sh的脚本，修改其中的版本信息。
#!/bin/bash ETCD_VER=v3.3.4 ETCD_DIR=etcd-download DOWNLOAD_URL=https://github.com/coreos/etcd/releases/download # Download mkdir ${ETCD_DIR} cd ${ETCD_DIR} wget ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz tar -xzvf etcd-${ETCD_VER}-linux-amd64.tar.gz # install cd etcd-${ETCD_VER}-linux-amd64 cp etcdctl /usr/local/bin/
2. etcdctl V3
使用etcdctlv3的版本时，需设置环境变量ETCDCTL_API=3。
export ETCDCTL_API=3 或者在`/etc/profile`文件中添加环境变量 vi /etc/profile ... export ETCDCTL_API=3 ... source /etc/profile
查看当前etcdctl的版本信息etcdctl version。
[root@k8s-dbg-master-1 etcd]# etcdctl version etcdctl version: 3.3.4 API version: 3.3
更多命令帮助可以查询etcdctl —help。
[root@k8s-dbg-master-1 etcd]# etcdctl --help NAME: etcdctl - A simple command line client for etcd3. USAGE: etcdctl VERSION: 3.3.4 API VERSION: 3.3 COMMANDS: get Gets the key or a range of keys put Puts the given key into the store del Removes the specified key or range of keys [key, range_end) txn Txn processes all the requests in one transaction compaction Compacts the event history in etcd alarm disarm Disarms all alarms alarm list Lists all alarms defrag Defragments the storage of the etcd members with given endpoints endpoint health Checks the healthiness of endpoints specified in `--endpoints` flag endpoint status Prints out the status of endpoints specified in `--endpoints` flag endpoint hashkv Prints the KV history hash for each endpoint in --endpoints move-leader Transfers leadership to another etcd cluster member. watch Watches events stream on keys or prefixes version Prints the version of etcdctl lease grant Creates leases lease revoke Revokes leases lease timetolive Get lease information lease list List all active leases lease keep-alive Keeps leases alive (renew) member add Adds a member into the cluster member remove Removes a member from the cluster member update Updates a member in the cluster member list Lists all members in the cluster snapshot save Stores an etcd node backend snapshot to a given file snapshot restore Restores an etcd member snapshot to an etcd directory snapshot status Gets backend snapshot status of a given file make-mirror Makes a mirror at the destination etcd cluster migrate Migrates keys in a v2 store to a mvcc store lock Acquires a named lock elect Observes and participates in leader election auth enable Enables authentication auth disable Disables authentication user add Adds a new user user delete Deletes a user user get Gets detailed information of a user user list Lists all users user passwd Changes password of user user grant-role Grants a role to a user user revoke-role Revokes a role from a user role add Adds a new role role delete Deletes a role role get Gets detailed information of a role role list Lists all roles role grant-permission Grants a key to a role role revoke-permission Revokes a key from a role check perf Check the performance of the etcd cluster help Help about any command OPTIONS: --cacert="" verify certificates of TLS-enabled secure servers using this CA bundle --cert="" identify secure client using this TLS certificate file --command-timeout=5s timeout for short running command (excluding dial timeout) --debug[=false] enable client-side debug logging --dial-timeout=2s dial timeout for client connections -d, --discovery-srv="" domain name to query for SRV records describing cluster endpoints --endpoints=[127.0.0.1:2379] gRPC endpoints --hex[=false] print byte strings as hex encoded strings --insecure-discovery[=true] accept insecure SRV records describing cluster endpoints --insecure-skip-tls-verify[=false] skip server certificate verification --insecure-transport[=true] disable transport security for client connections --keepalive-time=2s keepalive time for client connections --keepalive-timeout=6s keepalive timeout for client connections --key="" identify secure client using this TLS key file --user="" username[:password] for authentication (prompt if password is not supplied) -w, --write-out="simple" set the output format (fields, json, protobuf, simple, table)
3. etcdctl 常用命令
3.1. 指定etcd集群
HOST_1=10.240.0.17 HOST_2=10.240.0.18 HOST_3=10.240.0.19 ENDPOINTS=$HOST_1:2379,$HOST_2:2379,$HOST_3:2379 etcdctl --endpoints=$ENDPOINTS member list
执行查询时前缀是固定的，如下所示，使用这个前缀再加上etcd的查找命令即可成功查询：
ETCDCTL_API=3 /usr/local/bin/etcdctl --endpoints=https://127.0.0.1:2379 \ --cacert=/etc/kubernetes/pki/etcd/ca.crt \ --cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt \ --key=/etc/kubernetes/pki/etcd/healthcheck-client.key
命令行太长了，不容易记住，使用alias命令用来设置指令的别名。
alias etcdctl="ETCDCTL_API=3 /usr/local/bin/etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt --key=/etc/kubernetes/pki/etcd/healthcheck-client.key"
3.2. 增删改查
1、增
etcdctl --endpoints=$ENDPOINTS put foo "Hello World!"
2、查
etcdctl --endpoints=$ENDPOINTS get foo etcdctl --endpoints=$ENDPOINTS --write-out="json" get foo # 查看所有etcd的所有key，执行以下命令 etcdctl get / --prefix --keys-only # 查看某个POD信息 # 输出信息中，有少量不可见字符，这是因为etcd中存储的并不是json的原文，而是protocol buffer序列化后的数据，不过还是有部分内容是可读的； etcdctl get /registry/pods/default/nginx-deployment-6b474476c4-qsscn
基于相同前缀查找
etcdctl --endpoints=$ENDPOINTS put web1 value1 etcdctl --endpoints=$ENDPOINTS put web2 value2 etcdctl --endpoints=$ENDPOINTS put web3 value3 etcdctl --endpoints=$ENDPOINTS get web --prefix
列出所有的key
etcdctl --endpoints=$ENDPOINTS get / --prefix --keys-only
3、删
etcdctl --endpoints=$ENDPOINTS put key myvalue etcdctl --endpoints=$ENDPOINTS del key etcdctl --endpoints=$ENDPOINTS put k1 value1 etcdctl --endpoints=$ENDPOINTS put k2 value2 etcdctl --endpoints=$ENDPOINTS del k --prefix
3.3. 集群状态
集群状态主要是etcdctl endpoint status 和etcdctl endpoint health两条命令。
etcdctl --write-out=table --endpoints=$ENDPOINTS endpoint status +------------------+------------------+---------+---------+-----------+-----------+------------+ | ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX | +------------------+------------------+---------+---------+-----------+-----------+------------+ | 10.240.0.17:2379 | 4917a7ab173fabe7 | 3.0.0 | 45 kB | true | 4 | 16726 | | 10.240.0.18:2379 | 59796ba9cd1bcd72 | 3.0.0 | 45 kB | false | 4 | 16726 | | 10.240.0.19:2379 | 94df724b66343e6c | 3.0.0 | 45 kB | false | 4 | 16726 | +------------------+------------------+---------+---------+-----------+-----------+------------+ etcdctl --endpoints=$ENDPOINTS endpoint health 10.240.0.17:2379 is healthy: successfully committed proposal: took = 3.345431ms 10.240.0.19:2379 is healthy: successfully committed proposal: took = 3.767967ms 10.240.0.18:2379 is healthy: successfully committed proposal: took = 4.025451ms
3.4. 集群成员
跟集群成员相关的命令如下：
member add Adds a member into the cluster member remove Removes a member from the cluster member update Updates a member in the cluster member list Lists all members in the cluster
例如 etcdctl member list列出集群成员的命令。
etcdctl --endpoints=http://172.16.5.4:12379 member list -w table +-----------------+---------+-------+------------------------+-----------------------------------------------+ | ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS | +-----------------+---------+-------+------------------------+-----------------------------------------------+ | c856d92a82ba66a | started | etcd0 | http://172.16.5.4:2380 | http://172.16.5.4:2379,http://172.16.5.4:4001 | +-----------------+---------+-------+------------------------+-----------------------------------------------+
4. etcdctl get
使用etcdctl {command} --help可以查看具体命令的帮助信息。
# etcdctl get --help NAME: get - Gets the key or a range of keys USAGE: etcdctl get [options] <key> [range_end] OPTIONS: --consistency="l" Linearizable(l) or Serializable(s) --from-key[=false] Get keys that are greater than or equal to the given key using byte compare --keys-only[=false] Get only the keys --limit=0 Maximum number of results --order="" Order of results; ASCEND or DESCEND (ASCEND by default) --prefix[=false] Get keys with matching prefix --print-value-only[=false] Only write values when using the "simple" output format --rev=0 Specify the kv revision --sort-by="" Sort target; CREATE, KEY, MODIFY, VALUE, or VERSION GLOBAL OPTIONS: --cacert="" verify certificates of TLS-enabled secure servers using this CA bundle --cert="" identify secure client using this TLS certificate file --command-timeout=5s timeout for short running command (excluding dial timeout) --debug[=false] enable client-side debug logging --dial-timeout=2s dial timeout for client connections --endpoints=[127.0.0.1:2379] gRPC endpoints --hex[=false] print byte strings as hex encoded strings --insecure-skip-tls-verify[=false] skip server certificate verification --insecure-transport[=true] disable transport security for client connections --key="" identify secure client using this TLS key file --user="" username[:password] for authentication (prompt if password is not supplied) -w, --write-out="simple" set the output format (fields, json, protobuf, simple, table)
数据结构说明
/registry/apiregistration.k8s.io/apiservices/{版本}.{api名称}
包含 Kubernetes 中 API 服务的定义，因此我们可以在其中找到 Kubernetes 使用的所有现有核心 API，例如 /registry/apiregistration.k8s.io/apiservices/v1.batch或 /registry/apiregistration.k8s.io/apiservices/v1beta1.rbac。 authorization.k8s.io或自定义 API 定义（请参阅 https://github.com/kubernetes-incubator/apiserver-builder/blob/master/docs/concepts/aggregation.md）。您可以通过在 Etcd 中读取该键的值（您将获得人类可读的 json）或更友好的方式使用 kubectl get apiservice v1beta1.authorization.k8s.io -o json（与直接值相同）来获取有关 API 的信息等访问）
/registry/clusterroles/{角色名称}
包含 Kubernetes 中所有集群范围角色的定义，因此我们可以在那里找到 /registry/clusterroles/cluster-admin 或 /registry/clusterroles/system:kube-scheduler 之类的内容。Etcd 中的数据是人类可读但难以理解的——我们可以看到一些操作，如获取、修补、更新 API 的某些部分
/registry/clusterrolebindings/{实体名称}
包含集群范围内的角色和用户/组/服务帐户之间的绑定，因此我们可以在那里找到 /registry/clusterrolebindings/cluster-admin或 /registry/clusterrolebindings/kubeadm:kubelet-bootstrap 之类的东西。Etcd 中的数据是人类可读但难以理解的。
/registry/roles/{namespace}/{role name} 和 /registry/rolebindings/{namespace}/{entity name}
与集群角色/绑定中的故事相同，但受命名空间限制，例如 /registry/roles/kube-system/system:controller:token-cleaner
/registry/serviceaccounts/{namespace}/{name}
所有服务帐户的定义
/registry/configmaps/{命名空间}/{地图名称}
所有配置映射存储为 yamls
/registry/controllerrevisions/{namespace}/{pod}
我发现 ControllerRevision 资源用于在 DaemonSet 和 StatefulSet ( https://kubernetes.io/docs/tasks/manage-daemon/rollback-daemon-set/ ) 中提供回滚可能性。在 Etcd 中，我们可以找到 pod 规范的快照。
/registry/daemonsets/{namespace}/{name} 和 /registry/deployments/{namespace}/{name} 等。
在这些键下，Kubernetes 存储有关不同部署的信息，例如 DaemonSet、Deployment、ReplicaSet、Job 等。在部署的情况下有趣的是，我们看到那里描述了 last-applied-configuration https://kubernetes.io/docs/concepts/overview/对象管理-kubectl/declarative-config/#merge-patch-calculation
/registry/minions/{节点名称}
Kubernetes 节点以前被称为“minions”，所以在 Etcd 中的名称仍然没有改变。我们看到有大量数据描述节点，例如：
CPU内核
内存大小
kubelet 的状态：例如 kubelet 有足够的可用磁盘空间或 kubelet 有足够的 PID 可用
IP地址
主机名
Docker 版本
Docker image /registry/ranges/servicenodeportss 在节点上可用
/registry/namespaces/{namespace}
只是定义命名空间。还有特定命名空间的状态，如 Active 或 Terminating。
/registry/pods/{namespace}/{pod 名称}
集群中运行的每个 pod 的状态。包含很多信息，如 pod IP、挂载的卷、docker 映像等。
/registry/ranges/serviceips
服务的 CIDR
/registry/ranges/servicenodeports
暴露服务的端口范围
/registry/secrets/{namespace}/{pod}
集群中的所有秘密都以默认模式存储为纯文本。有关加密，请参阅 https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/
/registry/services/endpoints/{namespace}/{name}
服务定义。Kubernetes 计算特定服务选择了哪些 Pod，并将该信息存储在服务值中，以便我们可以在那里看到 Pod 的 IP 地址和名称。
文章参考：
https://coreos.com/etcd/docs/latest/demo.html
https://jakubbujny.wordpress.com/2018/09/02/what-stores-kubernetes-in-etcd/
