一、环境准备
1、K8s需要最低配置
- 一台兼容的 Linux 主机。Kubernetes 项目为基于 Debian 和 Red Hat 的 Linux 发行版以及一些不提供包管理器的发行版提供通用的指令。
- 每台机器 2 GB 或更多的 RAM（如果少于这个数字将会影响你应用的运行内存）。
- CPU 2 核心及以上。
- 集群中的所有机器的网络彼此均能相互连接（公网和内网都可以）。
- 节点之中不可以有重复的主机名、MAC 地址或 product_uuid。
- 开启机器上的某些端口。
- 禁用交换分区。为了保证 kubelet 正常工作，你必须禁用交换分区。
2、VMware创建安装Debian11虚拟机
在VMware可以创建3台以下规格的虚拟机，具体安装方法在网上有很多教程，Debian ISO镜像可以去以下链接下载https://www.debian.org/download
（1）下载系统镜像
（2）虚拟机规格：
IP |
hostName |
cpu |
内存 |
硬盘 |
备注 |
192.168.1.22 |
k8s-master |
2核 |
2G |
20G |
|
192.168.1.23 |
k8s-node1 |
2核 |
2G |
20G |
|
192.168.1.24 |
k8s-node2 |
2核 |
2G |
20G |
|
二、配置基本信息
1、配置host文件
k8s@k8s-master:~$ cat <
192.168.1.22 k8s-master
192.168.1.23 k8s-node1
192.168.1.23 k8s-node2
EOF
也可以使用vi 编辑 /etc/hosts，将配置信息添加到hosts中
2、禁用交互分区和设置时区
k8s@k8s-master:~$ sudo swapoff -a
k8s@k8s-master:~$ sudo sed -i 's/.*swap.*/#&/' /etc/fstab
# 设置系统时区为中国/上海（同步系统时区）
K8s@k8s-master:~$ sudo timedatectl set-timezone Asia/Shanghai
3、允许 iptables 检查桥接流量
k8s@k8s-master:~$cat br_netfilter
EOF
k8s@k8s-master:~$sudo modprobe overlay
k8s@k8s-master:~$sudo modprobe br_netfilter
# 设置所需的 sysctl 参数，参数在重新启动后保持不变
k8s@k8s-master:~$cat net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
# 应用 sysctl 参数而不重新启动
k8s@k8s-master:~$sudo sysctl --system
通过运行以下指令确认 br_netfilter 和 overlay 模块被加载：
k8s@k8s-master:~$ sudo lsmod | grep br_netfilter
k8s@k8s-master:~$ sudo lsmod | grep overlay
通过运行以下指令确认 net.bridge.bridge-nf-call-iptables、net.bridge.bridge-nf-call-ip6tables 和 net.ipv4.ip_forward 系统变量在你的 sysctl 配置中被设置为 1：
k8s@k8s-master:~$ sudo sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward
三、安装 Docker
1.卸载旧版本
在尝试安装之前卸载任何此类旧版本，然后在安装 新版本
k8s@k8s-master:~$sudo apt-get remove docker docker-engine docker.io containerd runc
2、安装软件包以允许 apt 通过 HTTPS 使用存储库
k8s@k8s-master:~$sudo apt-get update
k8s@k8s-master:~$sudo apt-get install apt-transport-https ca-certificates curl software-properties-common gnupg2
3、添加 Docker 的官方 GPG 密钥
k8s@k8s-master:~$ curl -fsSL https://download.docker.com/linux/debian/gpg|
sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
4、添加源
k8s@k8s-master:~$ echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian\
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
5、安装 docker 引擎
k8s@k8s-master:~$ sudo apt-get update
k8s@k8s-master:~$ sudo apt-get install docker-ce docker-ce-cli containerd.io
6、设置开机启动和启动docker
k8s@k8s-master:~$ sudo systemctl enable docker
k8s@k8s-master:~$ sudo systemctl start docker
7、使用阿里云ACR的加速服务
k8s@k8s-master: sudo mkdir -p /etc/docker
k8s@k8s-master: sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://e4vaaqhb.mirror.aliyuncs.com"]
}
EOF
8、配置 containerd
（1）新建 /etc/cri-containerd/config.toml
version = 2
# persistent data location
root = "/var/lib/cri-containerd"
# runtime state information
state = "/run/cri-containerd"
plugin_dir = ""
disabled_plugins = []
required_plugins = []
# set containerd's OOM score
oom_score = 0
[grpc]
address = "/run/cri-containerd/cri-containerd.sock"
tcp_address = ""
tcp_tls_cert = ""
tcp_tls_key = ""
# socket uid
uid = 0
# socket gid
gid = 0
max_recv_message_size = 16777216
max_send_message_size = 16777216
[debug]
address = ""
format = "json"
uid = 0
gid = 0
level = ""
[metrics]
address = "127.0.0.1:1338"
grpc_histogram = false
[cgroup]
path = ""
[timeouts]
"io.containerd.timeout.shim.cleanup" = "5s"
"io.containerd.timeout.shim.load" = "5s"
"io.containerd.timeout.shim.shutdown" = "3s"
"io.containerd.timeout.task.state" = "2s"
[plugins]
[plugins."io.containerd.gc.v1.scheduler"]
pause_threshold = 0.02
deletion_threshold = 0
mutation_threshold = 100
schedule_delay = "0s"
startup_delay = "100ms"
[plugins."io.containerd.grpc.v1.cri"]
disable_tcp_service = true
stream_server_address = "127.0.0.1"
stream_server_port = "0"
stream_idle_timeout = "4h0m0s"
enable_selinux = false
selinux_category_range = 1024
sandbox_image = "registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5"
stats_collect_period = 10
# systemd_cgroup = false
enable_tls_streaming = false
max_container_log_line_size = 16384
disable_cgroup = false
disable_apparmor = false
restrict_oom_score_adj = false
max_concurrent_downloads = 3
disable_proc_mount = false
unset_seccomp_profile = ""
tolerate_missing_hugetlb_controller = true
disable_hugetlb_controller = true
ignore_image_defined_volumes = false
[plugins."io.containerd.grpc.v1.cri".containerd]
snapshotter = "overlayfs"
default_runtime_name = "runc"
no_pivot = false
disable_snapshot_annotations = false
discard_unpacked_layers = false
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
pod_annotations = []
container_annotations = []
privileged_without_host_devices = false
base_runtime_spec = ""
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
# SystemdCgroup enables systemd cgroups.
SystemdCgroup = true
# BinaryName is the binary name of the runc binary.
# BinaryName = "runc"
# BinaryName = "crun"
# NoPivotRoot disables pivot root when creating a container.
# NoPivotRoot = false
# NoNewKeyring disables new keyring for the container.
# NoNewKeyring = false
# ShimCgroup places the shim in a cgroup.
# ShimCgroup = ""
# IoUid sets the I/O's pipes uid.
# IoUid = 0
# IoGid sets the I/O's pipes gid.
# IoGid = 0
# Root is the runc root directory.
Root = ""
# CriuPath is the criu binary path.
# CriuPath = ""
# CriuImagePath is the criu image path
# CriuImagePath = ""
# CriuWorkPath is the criu work path.
# CriuWorkPath = ""
[plugins."io.containerd.grpc.v1.cri".cni]
bin_dir = "/opt/cni/bin"
conf_dir = "/etc/cni/net.d"
max_conf_num = 1
conf_template = ""
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/cri-containerd/certs.d"
[plugins."io.containerd.grpc.v1.cri".registry.headers]
# Foo = ["bar"]
[plugins."io.containerd.grpc.v1.cri".image_decryption]
key_model = ""
[plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
tls_cert_file = ""
tls_key_file = ""
[plugins."io.containerd.internal.v1.opt"]
path = "/opt/cri-containerd"
[plugins."io.containerd.internal.v1.restart"]
interval = "10s"
[plugins."io.containerd.metadata.v1.bolt"]
content_sharing_policy = "shared"
[plugins."io.containerd.monitor.v1.cgroups"]
no_prometheus = false
[plugins."io.containerd.runtime.v2.task"]
platforms = ["linux/amd64"]
[plugins."io.containerd.service.v1.diff-service"]
default = ["walking"]
[plugins."io.containerd.snapshotter.v1.devmapper"]
root_path = ""
pool_name = ""
base_image_size = ""
async_remove = false
（2）新建 /etc/systemd/system/cri-containerd.service 文件
[Unit]
Description=containerd container runtime for kubernetes
Documentation=https://containerd.io
After=network.target local-fs.target
[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/bin/containerd --config //etc/cri-containerd/config.toml
Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999
[Install]
WantedBy=multi-user.target
9、重启 docker
k8s@k8s-master: sudo systemctl daemon-reload # 重新加载某个服务的配置文件，如果新安装了一个服务，归属于 systemctl 管理，要是新服务的服务程序配置文件生效，需重新加载。
k8s@k8s-master: sudo systemctl restart docker
10、将当前用户加入 docker 用户组
需要将当前用户加入 docker 用户组，否则进行相关操作会提示没有权限，也不建议使用 root 身份来操作docker
k8s@k8s-master:sudo usermod -aG docker $USER && newgrp docker
四、安装 kubeadm、kubelet和 kubectl
1、安装使用Kubernetes apt 仓库所需要的包
k8s@k8s-master:~$ sudo apt-get update
k8s@k8s-master:~$ sudo apt-get install -y apt-transport-https ca-certificates curl
2、下载 kubernetes 签名秘钥
由于官方 k8s 源在 google，国内无法访问，这里使用阿里云yum源
k8s@k8s-master:~$ curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
3、添加Kubernetes apt 仓库
k8s@k8s-master:~$ cat https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF
4、安装 kubelet、kubeadm 和 kubectl，并锁定其版本
k8s@k8s-master:~$ sudo apt-get update
k8s@k8s-master:~$ sudo apt-get install -y kubelet kubeadm kubectl
k8s@k8s-master:~$ sudo apt-mark hold kubelet kubeadm kubectl
5、添加开机启动并启动k8s服务
k8s@k8s-master:~$ sudo systemctl enable kubelet.service
k8s@k8s-master:~$ sudo systemctl start kubelet.service
五、部署 kubernetes
1、初始化master
k8s@k8s-master:~$sudo kubeadm init --kubernetes-version=v1.26.2 --image-repository registry.aliyuncs.com/google_containers --apiserver-advertise-address=192.168.1.22 --service-cidr=10.245.0.0/12 --pod-network-cidr=10.244.0.0/16 --cri-socket /run/cri-containerd/cri-containerd.sock
2、加入node节点
k8s@k8s-node1:~$sudo kubeadm join 192.168.1.22:6443 --token 3kivfl.yy6t91pa3omz1yfz --discovery-token-ca-cert-hash sha256:74a5a255bdb3dadd7be65cf171af538714a07b7d28dc5d03ed25e86ba8ae8c5b --cri-socket /run/cri-containerd/cri-containerd.sock
sha256:74a5a255bdb3dadd7be65cf171af538714a07b7d28dc5d03ed25e86ba8ae8c5b 是master生成的token,默认 token 有效期为24小时，当过期之后，该token就不可用了。这时就需要重新创建 token，操作如下
k8s@k8s-master:~$ sudo kubeadm token create --print-join-command
3、查看node节点
运行 kubectl get nodes 命令查看：
k8s@k8s-master:/etc/kubernetes$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master NotReady control-plane 6d21h v1.26.2
k8s-node1 NotReady 6d21h v1.26.2
k8s-node2 NotReady 6d21h v1.26.2
4、安装 CNI 网络插件
此时，虽然 Maste 节点可以检测到 node 节点，但 STATUS 都还是 NotReady 状态，这里需要看装 CNI节点来实现网络访问
安装 calico 插件
# 这里选择的是 calico 插件
k8s@k8s-master:~$ sudo kubectl apply -f https://docs.projectcalico.org/v3.20/manifests/calico.yaml
k8s@k8s-master:/etc/kubernetes$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default nginx-748c667d99-6vs77 1/1 Running 0 4h15m
kube-system calico-kube-controllers-6d674b5f78-jc47d 0/1 Error 62 (5m17s ago) 6d21h
kube-system calico-node-7cqmq 0/1 Running 0 6d21h
kube-system calico-node-plw6z 0/1 Running 0 6d21h
kube-system calico-node-srnvw 0/1 Running 0 6d21h
kube-system coredns-5bbd96d687-k2dgb 0/1 CrashLoopBackOff 68 (2m58s ago) 6d21h
kube-system coredns-5bbd96d687-lmfp8 0/1 CrashLoopBackOff 68 (2m58s ago) 6d21h
kube-system etcd-k8s-master 1/1 Running 0 6d21h
kube-system kube-apiserver-k8s-master 1/1 Running 0 6d21h
kube-system kube-controller-manager-k8s-master 1/1 Running 0 6d21h
kube-system kube-proxy-hzl8b 1/1 Running 0 6d21h
kube-system kube-proxy-jvcjf 1/1 Running 0 6d21h
kube-system kube-proxy-v42v7 1/1 Running 0 6d21h
kube-system kube-scheduler-k8s-master 1/1 Running 0 6d21h
k8s@k8s-master:/etc/kubernetes$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready control-plane 6d21h v1.26.2
k8s-node1 Ready 6d21h v1.26.2
k8s-node2 Ready 6d21h v1.26.2
此时，各个节点状态已就绪
六、测试 kubernetes 集群
1、拉取一个nginx镜像，并创建 nginx Pod
k8s@k8s-master:~$ kubectl create deployment nginx --image=nginx
2、查看 nginx Pod 状态：当 STATUS 为 Running 即表示创建完成
k8s@k8s-master:~$ kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-748c667d99-6vs77 0/1 ContainerCreating 0 20s
k8s@k8s-master:~$ kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-748c667d99-6vs77 1/1 Running 0 4h21m
3、对外暴露 nginx 端口
k8s@k8s-master:~$ kubectl expose deployment nginx --port=80 --type=NodePort
service/nginx exposed
4、查看对外暴露端口状态
k8s@k8s-master:/etc/kubernetes$ kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/nginx-748c667d99-6vs77 1/1 Running 0 2m24s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.240.0.1 443/TCP 6d17h
service/nginx NodePort 10.253.103.236 80:31369/TCP 44s
5、查看结果
在外网中以任一 node（192.168.1.23、192.168.1.24） 节点加上对外暴露端口 31369访问可以看到如下结果：
