一、集群环境规划配置
生产环境不要使用一主多从，要使用多主多从。这里使用三台主机进行测试一台Master（172.16.20.111），两台Node（172.16.20.112和172.16.20.113）
1、设置主机名
CentOS7安装完成之后，设置固定ip，三台主机做相同设置
vi /etc/sysconfig/network-scripts/ifcfg-ens33 #在最下面ONBOOT改为yes，新增固定地址IPADDR，172.16.20.111，172.16.20.112，172.16.20.113 ONBOOT=yes IPADDR=172.16.20.111
三台主机ip分别设置好之后，修改hosts文件，设置主机名
#master 机器上执行 hostnamectl set-hostname master #node1 机器上执行 hostnamectl set-hostname node1 #node2 机器上执行 hostnamectl set-hostname node2
vi /etc/hosts 172.16.20.111 master 172.16.20.112 node1 172.16.20.113 node2
2、时间同步
开启chronyd服务
systemctl start chronyd
设置开机启动
systemctl enable chronyd
测试
date
3、禁用firewalld和iptables（测试环境）
systemctl stop firewalld systemctl disable firewalld systemctl stop iptables systemctl disable iptables
4、禁用selinux
vi /etc/selinux/config SELINUX=disabled
5、禁用swap分区
注释掉 /dev/mapper/centos-swap swap
vi /etc/fstab # 注释掉 # /dev/mapper/centos-swap swap
6、修改linux的内核参数
vi /etc/sysctl.d/kubernetes.conf net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 #重新加载配置 sysctl -p #加载网桥过滤模块 modprobe br_netfilter #查看网桥过滤模块 lsmod | grep br_netfilter
7、配置ipvs
安装ipset和ipvsadm
yum install ipset ipvsadm -y
添加需要加载的模块（整个执行）
cat <<EOF> /etc/sysconfig/modules/ipvs.modules #!/bin/bash modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack_ipv4 EOF
添加执行权限
chmod +x /etc/sysconfig/modules/ipvs.modules
执行脚本
/bin/bash /etc/sysconfig/modules/ipvs.modules
查看是否加载成功
lsmod | grep -e -ip_vs -e nf_conntrack_ipv4
以上完成设置之后，一定要执行重启使配置生效
reboot
二、Docker环境安装配置
1、安装依赖
docker依赖于系统的一些必要的工具:
yum install -y yum-utils device-mapper-persistent-data lvm2
2、添加软件源
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo yum clean all yum makecache fast
3、安装docker-ce
#查看可以安装的docker版本 yum list docker-ce --showduplicates #选择安装需要的版本，直接安装最新版，可以执行 yum -y install docker-ce yum install --setopt=obsoletes=0 docker-ce-19.03.13-3.el7 -y
4、启动服务
#通过systemctl启动服务 systemctl start docker #通过systemctl设置开机启动 systemctl enable docker
5、查看安装版本
启动服务使用docker version查看一下当前的版本:
docker version
6、 配置镜像加速
通过修改daemon配置文件/etc/docker/daemon.json加速，如果使用k8s，这里一定要设置 "exec-opts": ["native.cgroupdriver=systemd"]。 "insecure-registries" : ["172.16.20.175"]配置是可以通过http从我们的harbor上拉取数据。
vi /etc/docker/daemon.json { "exec-opts": ["native.cgroupdriver=systemd"], "log-driver": "json-file", "log-opts": { "max-size": "100m" }, "registry-mirrors": ["https://eiov0s1n.mirror.aliyuncs.com"], "insecure-registries" : ["172.16.20.175"] } sudo systemctl daemon-reload && sudo systemctl restart docker
7、安装docker-compose
如果网速太慢，可以直接到 https://github.com/docker/compose/releases 选择对应的版本进行下载，然后上传到服务器/usr/local/bin/目录。
sudo curl -L "https://github.com/docker/compose/releases/download/v2.0.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose
注意：（非必须设置）开启Docker远程访问 (这里不是必须开启的，生产环境不要开启，开启之后，可以在开发环境直连docker)
vi /lib/systemd/system/docker.service
修改ExecStart，添加 -H tcp://0.0.0.0:2375
ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375 --containerd=/run/containerd/containerd.sock
修改后执行以下命令：
systemctl daemon-reload && service docker restart
测试是否能够连得上：
curl http://localhost:2375/version
三、Harbor私有镜像仓库安装配置（重新设置一台服务器172.16.20.175，不要放在K8S的主从服务器上）
首先需要按照前面的步骤，在环境上安装Docker，才能安装Harbor。
1、选择合适的版本进行下载，下载地址：
https://github.com/goharbor/harbor/releases
2、解压
tar -zxf harbor-offline-installer-v2.2.4.tgz
3、配置
cd harbor mv harbor.yml.tmpl harbor.yml vi harbor.yml
4、将hostname改为当前服务器地址，注释掉https配置。
...... # The IP address or hostname to access admin UI and registry service. # DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients. hostname: 172.16.20.175 # http related config http: # port for http, default is 80. If https enabled, this port will redirect to https port port: 80 # https related config #https: # https port for harbor, default is 443 # port: 443 # The path of cert and key files for nginx # certificate: /your/certificate/path # private_key: /your/private/key/path ......
5、执行安装命令
mkdir /var/log/harbor/ ./install.sh
6、查看安装是否成功
[root@localhost harbor]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES de1b702759e7 goharbor/harbor-jobservice:v2.2.4 "/harbor/entrypoint.…" 13 seconds ago Up 9 seconds (health: starting) harbor-jobservice 55b465d07157 goharbor/nginx-photon:v2.2.4 "nginx -g 'daemon of…" 13 seconds ago Up 9 seconds (health: starting) 0.0.0.0:80->8080/tcp, :::80->8080/tcp nginx d52f5557fa73 goharbor/harbor-core:v2.2.4 "/harbor/entrypoint.…" 13 seconds ago Up 10 seconds (health: starting) harbor-core 4ba09aded494 goharbor/harbor-db:v2.2.4 "/docker-entrypoint.…" 13 seconds ago Up 11 seconds (health: starting) harbor-db 647f6f46e029 goharbor/harbor-portal:v2.2.4 "nginx -g 'daemon of…" 13 seconds ago Up 11 seconds (health: starting) harbor-portal 70251c4e234f goharbor/redis-photon:v2.2.4 "redis-server /etc/r…" 13 seconds ago Up 11 seconds (health: starting) redis 21a5c408afff goharbor/harbor-registryctl:v2.2.4 "/home/harbor/start.…" 13 seconds ago Up 11 seconds (health: starting) registryctl b0937800f88b goharbor/registry-photon:v2.2.4 "/home/harbor/entryp…" 13 seconds ago Up 11 seconds (health: starting) registry d899e377e02b goharbor/harbor-log:v2.2.4 "/bin/sh -c /usr/loc…" 13 seconds ago Up 12 seconds (health: starting) 127.0.0.1:1514->10514/tcp harbor-log
7、harbor的启动停止命令
docker-compose down #停止 docker-compose up -d #启动
8、访问harbor管理台地址，上面配置的hostname, http://172.16.20.175 （默认用户名/密码： admin/Harbor12345）：
三、Kubernetes安装配置
1、切换镜像源
cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF
2、安装kubeadm、kubelet和kubectl
yum install -y kubelet kubeadm kubectl
3、配置kubelet的cgroup
vi /etc/sysconfig/kubelet
KUBELET_CGROUP_ARGS="--cgroup-driver=systemd" KUBE_PROXY_MODE="ipvs"
4、启动kubelet并设置开机启动
systemctl start kubelet && systemctl enable kubelet
5、初始化k8s集群（只在Master执行）
初始化
kubeadm init --kubernetes-version=v1.22.3 \ --apiserver-advertise-address=172.16.20.111 \ --image-repository registry.aliyuncs.com/google_containers \ --service-cidr=10.20.0.0/16 --pod-network-cidr=10.222.0.0/16
初始化成功
创建必要文件
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
6、加入集群（只在Node节点执行）
在Node节点（172.16.20.112和172.16.20.113）运行上一步初始化成功后显示的加入集群命令
kubeadm join 172.16.20.111:6443 --token fgf380.einr7if1eb838mpe \ --discovery-token-ca-cert-hash sha256:fa5a6a2ff8996b09effbf599aac70505b49f35c5bca610d6b5511886383878f7
在Master查看集群状态
[root@master ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION master NotReady control-plane,master 2m54s v1.22.3 node1 NotReady <none> 68s v1.22.3 node2 NotReady <none> 30s v1.22.3
7、安装网络插件（只在Master执行）
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
镜像加速：修改kube-flannel.yml文件，将quay.io/coreos/flannel:v0.15.0 改为 quay.mirrors.ustc.edu.cn/coreos/flannel:v0.15.0
执行安装
kubectl apply -f kube-flannel.yml
再次查看集群状态,（需要等待一段时间大概1-2分钟）发现STATUS都是Ready。
[root@master ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION master Ready control-plane,master 42m v1.22.3 node1 Ready <none> 40m v1.22.3 node2 Ready <none> 39m v1.22.3
8、集群测试
使用kubectl安装部署nginx服务
kubectl create deployment nginx --image=nginx --replicas=1 kubectl expose deploy nginx --port=80 --target-port=80 --type=NodePort
查看服务
[root@master ~]# kubectl get pod,svc NAME READY STATUS RESTARTS AGE pod/nginx-6799fc88d8-z5tm8 1/1 Running 0 26s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubernetes ClusterIP 10.20.0.1 <none> 443/TCP 68m service/nginx NodePort 10.20.17.199 <none> 80:32605/TCP 9s
服务显示service/nginx的PORT(S)为80:32605/TCP， 我们在浏览器中访问主从地址的32605端口，查看nginx是否运行
成功后显示如下界面：
Nginx
9、安装Kubernetes管理界面Dashboard
Kubernetes可以通过命令行工具kubectl完成所需要的操作，同时也提供了方便操作的管理控制界面，用户可以用 Kubernetes Dashboard 部署容器化的应用、监控应用的状态、执行故障排查任务以及管理 Kubernetes 各种资源。
1、下载安装配置文件recommended.yaml ,注意在https://github.com/kubernetes/dashboard/releases查看Kubernetes 和 Kubernetes Dashboard的版本对应关系。
版本对应关系
# 执行下载 wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.4.0/aio/deploy/recommended.yaml
2、修改配置信息，在service下添加 type: NodePort和nodePort: 30010
vi recommended.yaml
...... kind: Service apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard spec: # 新增 nodeName: Master # 新增 type: NodePort ports: - port: 443 targetPort: 8443 # 新增 nodePort: 30010 ......
注释掉以下信息，否则不能安装到master服务器
# Comment the following tolerations if Dashboard must not be deployed on master #tolerations: # - key: node-role.kubernetes.io/master # effect: NoSchedule
新增nodeName: master，安装到master服务器
...... kind: Deployment apiVersion: apps/v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard spec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s-app: kubernetes-dashboard template: metadata: labels: k8s-app: kubernetes-dashboard spec: nodeName: master containers: - name: kubernetes-dashboard image: kubernetesui/dashboard:v2.4.0 imagePullPolicy: Always ......
3、执行安装部署命令
kubectl apply -f recommended.yaml
4、查看运行状态命令，可以看到service/kubernetes-dashboard 已运行，访问端口为30010
[root@master ~]# kubectl get pod,svc -n kubernetes-dashboard NAME READY STATUS RESTARTS AGE pod/dashboard-metrics-scraper-c45b7869d-6k87n 0/1 ContainerCreating 0 10s pod/kubernetes-dashboard-576cb95f94-zfvc9 0/1 ContainerCreating 0 10s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/dashboard-metrics-scraper ClusterIP 10.20.222.83 <none> 8000/TCP 10s service/kubernetes-dashboard NodePort 10.20.201.182 <none> 443:30010/TCP 10s
5、创建访问Kubernetes Dashboard的账号
kubectl create serviceaccount dashboard-admin -n kubernetes-dashboard kubectl create clusterrolebinding dashboard-admin-rb --clusterrole=cluster-admin --serviceaccount=kubernetes-dashboard:dashboard-admin
6、查询访问Kubernetes Dashboard的token
[root@master ~]# kubectl get secrets -n kubernetes-dashboard | grep dashboard-admin dashboard-admin-token-84gg6 kubernetes.io/service-account-token 3 64s [root@master ~]# kubectl describe secrets dashboard-admin-token-84gg6 -n kubernetes-dashboard Name: dashboard-admin-token-84gg6 Namespace: kubernetes-dashboard Labels: <none> Annotations: kubernetes.io/service-account.name: dashboard-admin kubernetes.io/service-account.uid: 2d93a589-6b0b-4ed6-adc3-9a2eeb5d1311 Type: kubernetes.io/service-account-token Data ==== ca.crt: 1099 bytes namespace: 20 bytes token: eyJhbGciOiJSUzI1NiIsImtpZCI6ImRmbVVfRy15QzdfUUF4ZmFuREZMc3dvd0IxQ3ItZm5SdHVZRVhXV3JpZGcifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tODRnZzYiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMmQ5M2E1ODktNmIwYi00ZWQ2LWFkYzMtOWEyZWViNWQxMzExIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmVybmV0ZXMtZGFzaGJvYXJkOmRhc2hib2FyZC1hZG1pbiJ9.xsDBLeZdn7IO0Btpb4LlCD1RQ2VYsXXPa-bir91VXIqRrL1BewYAyFfZtxU-8peU8KebaJiRIaUeF813x6WbGG9QKynL1fTARN5XoH-arkBTVlcjHQ5GBziLDE-KU255veVqORF7J5XtB38Ke2n2pi8tnnUUS_bIJpMTF1s-hV0aLlqUzt3PauPmDshtoerz4iafWK0u9oWBASQDPPoE8IWYU1KmSkUNtoGzf0c9vpdlUw4j0UZE4-zSoMF_XkrfQDLD32LrG56Wgpr6E8SeipKRfgXvx7ExD54b8Lq9DyAltr_nQVvRicIEiQGdbeCu9dwzGyhg-cDucULTx7TUgA
7、在页面访问Kubernetes Dashboard，注意一定要使用https，https://172.16.20.111:30010 ，输入token登录成功后就进入了后台管理界面，原先命令行的操作就可以在管理界面进操作了
输入token
管理界面
四、GitLab安装配置
GitLab是可以部署在本地环境的Git项目仓库，这里介绍如何安装使用，在开发过程中我们将代码上传到本地仓库，然后Jenkins从仓库中拉取代码打包部署。
1、下载需要的安装包，下载地址 https://packages.gitlab.com/gitlab/gitlab-ce/ ，我们这里下载最新版gitlab-ce-14.4.1-ce.0.el7.x86_64.rpm，当然在项目开发中需要根据自己的需求选择稳定版本
image.png
2、点击需要安装的版本，会提示安装命令，按照上面提示的命令进行安装即可
curl -s https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | sudo bash sudo yum install gitlab-ce-14.4.1-ce.0.el7.x86_64
3、配置并启动Gitlab
gitlab-ctl reconfigure
4、查看Gitlab状态
gitlab-ctl status
5、设置初始登录密码
cd /opt/gitlab/bin sudo ./gitlab-rails console # 进入控制台之后执行 u=User.where(id:1).first u.password='root1234' u.password_confirmation='root1234' u.save! quit
5、浏览器访问服务器地址，默认是80端口，所以直接访问即可，在登录界面输入我们上面设置的密码root/root1234。
image.png
image.png
6、设置界面为中文
User Settings ----> Preferences ----> Language ----> 简体中文 ----> 刷新界面
image.png
7、Gitlab常用命令
gitlab-ctl stop gitlab-ctl start gitlab-ctl restart
五、使用Docker安装配置Jenkins+Sonar（代码质量检查）
实际项目应用开发过程中，单独为SpringCloud工程部署一台运维服务器，不要安装在Kubernetes服务器上，同样按照上面的步骤安装docker和docker-compose，然后使用docker-compose构建Jenkins和Sonar。
1、创建宿主机挂载目录并赋权
mkdir -p /data/docker/ci/nexus /data/docker/ci/jenkins/lib /data/docker/ci/jenkins/home /data/docker/ci/sonarqube /data/docker/ci/postgresql chmod -R 777 /data/docker/ci/nexus /data/docker/ci/jenkins/lib /data/docker/ci/jenkins/home /data/docker/ci/sonarqube /data/docker/ci/postgresql
2、新建Jenkins+Sonar安装脚本jenkins-compose.yml脚本，这里的Jenkins使用的是Docker官方推荐的镜像jenkinsci/blueocean，在实际使用中发现，即使不修改插件下载地址，也可以下载插件，所以比较推荐这个镜像。
version: '3' networks: prodnetwork: driver: bridge services: sonardb: image: postgres:12.2 restart: always ports: - "5433:5432" networks: - prodnetwork volumes: - /data/docker/ci/postgresql:/var/lib/postgresql environment: - POSTGRES_USER=sonar - POSTGRES_PASSWORD=sonar sonar: image: sonarqube:8.2-community restart: always ports: - "19000:9000" - "19092:9092" networks: - prodnetwork depends_on: - sonardb volumes: - /data/docker/ci/sonarqube/conf:/opt/sonarqube/conf - /data/docker/ci/sonarqube/data:/opt/sonarqube/data - /data/docker/ci/sonarqube/logs:/opt/sonarqube/logs - /data/docker/ci/sonarqube/extension:/opt/sonarqube/extensions - /data/docker/ci/sonarqube/bundled-plugins:/opt/sonarqube/lib/bundled-plugins environment: - TZ=Asia/Shanghai - SONARQUBE_JDBC_URL=jdbc:postgresql://sonardb:5432/sonar - SONARQUBE_JDBC_USERNAME=sonar - SONARQUBE_JDBC_PASSWORD=sonar nexus: image: sonatype/nexus3 restart: always ports: - "18081:8081" networks: - prodnetwork volumes: - /data/docker/ci/nexus:/nexus-data jenkins: image: jenkinsci/blueocean user: root restart: always ports: - "18080:8080" networks: - prodnetwork volumes: - /var/run/docker.sock:/var/run/docker.sock - /etc/localtime:/etc/localtime:ro - $HOME/.ssh:/root/.ssh - /data/docker/ci/jenkins/lib:/var/lib/jenkins/ - /usr/bin/docker:/usr/bin/docker - /data/docker/ci/jenkins/home:/var/jenkins_home depends_on: - nexus - sonar environment: - NEXUS_PORT=8081 - SONAR_PORT=9000 - SONAR_DB_PORT=5432 cap_add: - ALL
3、在jenkins-compose.yml文件所在目录下执行安装启动命令
docker-compose -f jenkins-compose.yml up -d
安装成功后，展示以下信息
[+] Running 5/5 ⠿ Network root_prodnetwork Created 0.0s ⠿ Container root-sonardb-1 Started 1.0s ⠿ Container root-nexus-1 Started 1.0s ⠿ Container root-sonar-1 Started 2.1s ⠿ Container root-jenkins-1 Started 4.2s
4、查看服务的启动情况
[root@localhost ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 52779025a83e jenkins/jenkins:lts "/sbin/tini -- /usr/…" 4 minutes ago Up 3 minutes 50000/tcp, 0.0.0.0:18080->8080/tcp, :::18080->8080/tcp root-jenkins-1 2f5fbc25de58 sonarqube:8.2-community "./bin/run.sh" 4 minutes ago Restarting (0) 21 seconds ago root-sonar-1 4248a8ba71d8 sonatype/nexus3 "sh -c ${SONATYPE_DI…" 4 minutes ago Up 4 minutes 0.0.0.0:18081->8081/tcp, :::18081->8081/tcp root-nexus-1 719623c4206b postgres:12.2 "docker-entrypoint.s…" 4 minutes ago Up 4 minutes 0.0.0.0:5433->5432/tcp, :::5433->5432/tcp root-sonardb-1 2b6852a57cc2 goharbor/harbor-jobservice:v2.2.4 "/harbor/entrypoint.…" 5 days ago Up 29 seconds (health: starting) harbor-jobservice ebf2dea994fb goharbor/nginx-photon:v2.2.4 "nginx -g 'daemon of…" 5 days ago Restarting (1) 46 seconds ago nginx adfaa287f23b goharbor/harbor-registryctl:v2.2.4 "/home/harbor/start.…" 5 days ago Up 7 minutes (healthy) registryctl 8e5bcca3aaa1 goharbor/harbor-db:v2.2.4 "/docker-entrypoint.…" 5 days ago Up 7 minutes (healthy) harbor-db ebe845e020dc goharbor/harbor-portal:v2.2.4 "nginx -g 'daemon of…" 5 days ago Up 7 minutes (healthy) harbor-portal 68263dea2cfc goharbor/harbor-log:v2.2.4 "/bin/sh -c /usr/loc…" 5 days ago Up 7 minutes (healthy) 127.0.0.1:1514->10514/tcp harbor-log
我们发现 jenkins端口映射到了18081 ，但是sonarqube没有启动，查看日志发现
sonarqube文件夹没有权限访问，日志上显示容器目录的权限不够，但实际是宿主机的权限不够，这里需要给宿主机赋予权限
chmod 777 /data/docker/ci/sonarqube/logs chmod 777 /data/docker/ci/sonarqube/bundled-plugins chmod 777 /data/docker/ci/sonarqube/conf chmod 777 /data/docker/ci/sonarqube/data chmod 777 /data/docker/ci/sonarqube/extension
执行重启命令
docker-compose -f jenkins-compose.yml restart
再次使用命令查看服务启动情况，就可以看到jenkins映射到18081，sonarqube映射到19000端口，我们在浏览器就可以访问jenkins和sonarqube的后台界面了
jenkins
sonarqube
5、Jenkins登录初始化
从Jenkins的登录界面提示可以知道，默认密码路径
为/var/jenkins_home/secrets/initialAdminPassword，这里显示的事Docker容器内部的路径，实际对应我们上面服务器设置的路径为/data/docker/ci/jenkins/home/secrets/initialAdminPassword ,我们打开这个文件并输入密码就可以进入Jenkins管理界面
插件管理界面
6、选择安装推荐插件，安装完成之后，根据提示进行下一步操作，直到进入管理后台界面
image.png
image.png
image.png
