- 1.打包springboot镜像
- 2.在kubernete上发布镜像
- 3.测试
在之前的文章中，我讲了使用kubeadm从0到1搭建kubernete集群，今天我们来聊一下如何在这套k8s集群上部署springboot应用。首先说明一下，我们这次集群的网络使用flannel网络，master节点启动命令如下:
kubeadm init --image-repository registry.aliyuncs.com/google_containers --kubernetes-version=v1.17.3 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address 192.168.59.132
上面的参数 --pod-network-cidr=10.244.0.0/16
就是指明集群使用flannel网络，切记，不然部署springboot后，容器访问外面网络会有问题。
再来看一下这次实验使用的集群环境，如下图，我在个人笔记本上安装了mysql数据库，ip地址是192.168.59.1，同时安装了2个vmware虚拟机，ip地址分别是192.168.59.132和192.168.59.138，kubernete集群就部署在这2个虚机上，192.168.59.132部署着master节点，192.168.59.138部署着worker1节点
这次使用的springboot源码地址：
https://github.com/jinjunzhu/spring-boot-mybatis
1.打包springboot镜像
1.打包springboot镜像
这儿遇到问题可以查看我之前的文章Linux下docker制作springboot应用镜像
编译jar包
mvn clean -Dmaven.test.skip=true install
编译后上传到虚机，编写DockerFile文件
FROM openjdk:8-jdk-alpine VOLUME /tmp ADD spring-boot-mybatis-1.0-SNAPSHOT.jar springboot-mybatis.jar ENTRYPOINT ["java","-jar","springboot-mybatis.jar"]
执行打包镜像命令，这里我打包的版本号是1.2，每次修改代码后都要升级版本号，不然上传dockerhub不会更新
docker build -t zjj2006forever/springboot-mybatis:1.2 .
之后下面2个命令登录dockerhub并且上传
docker login docker push zjj2006forever/springboot-mybatis:1.2 [root@master k8s]# docker push zjj2006forever/springboot-mybatis:1.2 The push refers to a repository [docker.io/zjj2006forever/springboot-mybatis] a3ce9cf52624: Pushed ceaf9e1ebef5: Pushed 9b9b7f3d56a0: Pushed f1b5933fe4b5: Pushed 1.0: digest: sha256:8d37516e910515b0e336ecc29befaa0f9240ebd92a3c22af8b90846476644420 size: 1159
2.在kubernete上发布镜像
编写yaml文件，如下：
apiVersion: apps/v1 kind: Deployment metadata: name: springboot-mybatis-deployment spec: selector: matchLabels: app: springboot-mybatis replicas: 2 template: metadata: labels: app: springboot-mybatis spec: containers: - name: spingboot-mybatis imagePullPolicy: IfNotPresent #imagePullPolicy: Never image: zjj2006forever/springboot-mybatis:1.2 ports: - containerPort: 8300
上面的yaml文件中，我指定了镜像名称是我刚刚打包的镜像：image: zjj2006forever/springboot-mybatis:1.2，副本数量是2（replicas: 2），
这样部署的时候会生成2个pod，如果提供服务的时候如果一个pod挂了，集群会再创建一个新的出来，一直保持有2个pod。
kubectl apply -f springboot-mybatis.yaml
输出下面提示后我们看一下pod状态
deployment.apps/springboot-mybatis-deployment created
[root@master k8s]# kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE default springboot-mybatis-deployment-9f5b78b76-2qtfg 1/1 Running 0 9s default springboot-mybatis-deployment-9f5b78b76-tzf4x 1/1 Running 0 9s kube-system coredns-9d85f5447-7k44c 1/1 Running 0 30m kube-system coredns-9d85f5447-sh24j 1/1 Running 0 30m kube-system etcd-master 1/1 Running 0 30m kube-system kube-apiserver-master 1/1 Running 0 30m kube-system kube-controller-manager-master 1/1 Running 0 30m kube-system kube-flannel-ds-amd64-rd842 1/1 Running 0 16m kube-system kube-flannel-ds-amd64-tdkwx 1/1 Running 0 16m kube-system kube-proxy-d6qqg 1/1 Running 0 25m kube-system kube-proxy-pm7cc 1/1 Running 0 30m kube-system kube-scheduler-master 1/1 Running 0 30m
再看一下pod启动日志
[root@master k8s]# kubectl logs -f springboot-mybatis-deployment-9f5b78b76-2qtfg . ____ _ __ _ _ /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \ ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \ \\/ ___)| |_)| | | | | || (_| | ) ) ) ) ' |____| .__|_| |_|_| |_\__, | / / / / =========|_|==============|___/=/_/_/_/ :: Spring Boot :: (v2.1.6.RELEASE) 2020-06-13 08:25:15,002 [main] [INFO] boot.Application - Starting Application v1.0-SNAPSHOT on springboot-mybatis-deployment-9f5b78b76-2qtfg with PID 1 (/springboot-mybatis.jar started by root in /) 2020-06-13 08:25:15,006 [main] [INFO] boot.Application - The following profiles are active: dev 2020-06-13 08:25:20,545 [main] [INFO] org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker - Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration$$EnhancerBySpringCGLIB$$c636a19f] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2020-06-13 08:25:21,688 [main] [INFO] org.springframework.boot.web.embedded.tomcat.TomcatWebServer - Tomcat initialized with port(s): 8083 (http) 2020-06-13 08:25:21,723 [main] [INFO] org.apache.coyote.http11.Http11NioProtocol - Initializing ProtocolHandler ["http-nio-8083"] 2020-06-13 08:25:21,839 [main] [INFO] org.apache.catalina.core.StandardService - Starting service [Tomcat] 2020-06-13 08:25:21,840 [main] [INFO] org.apache.catalina.core.StandardEngine - Starting Servlet engine: [Apache Tomcat/9.0.21] 2020-06-13 08:25:22,260 [main] [INFO] org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/] - Initializing Spring embedded WebApplicationContext 2020-06-13 08:25:22,261 [main] [INFO] org.springframework.web.context.ContextLoader - Root WebApplicationContext: initialization completed in 6985 ms 2020-06-13 08:25:25,220 [main] [INFO] springfox.documentation.spring.web.PropertySourcedRequestMappingHandlerMapping - Mapped URL path [/v2/api-docs] onto method [public org.springframework.http.ResponseEntity<springfox.documentation.spring.web.json.Json> springfox.documentation.swagger2.web.Swagger2Controller.getDocumentation(java.lang.String,javax.servlet.http.HttpServletRequest)] 2020-06-13 08:25:25,631 [main] [INFO] org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor - Initializing ExecutorService 'applicationTaskExecutor' 2020-06-13 08:25:27,595 [main] [INFO] springfox.documentation.spring.web.plugins.DocumentationPluginsBootstrapper - Context refreshed 2020-06-13 08:25:27,710 [main] [INFO] springfox.documentation.spring.web.plugins.DocumentationPluginsBootstrapper - Found 1 custom documentation plugin(s) 2020-06-13 08:25:27,857 [main] [INFO] springfox.documentation.spring.web.scanners.ApiListingReferenceScanner - Scanning for api listing references 2020-06-13 08:25:28,755 [main] [INFO] org.apache.coyote.http11.Http11NioProtocol - Starting ProtocolHandler ["http-nio-8083"] 2020-06-13 08:25:28,956 [main] [INFO] org.springframework.boot.web.embedded.tomcat.TomcatWebServer - Tomcat started on port(s): 8083 (http) with context path '' 2020-06-13 08:25:29,253 [main] [INFO] org.springframework.boot.web.embedded.tomcat.TomcatWebServer - Tomcat initialized with port(s): 18082 (http) 2020-06-13 08:25:29,255 [main] [INFO] org.apache.coyote.http11.Http11NioProtocol - Initializing ProtocolHandler ["http-nio-127.0.0.1-18082"] 2020-06-13 08:25:29,289 [main] [INFO] org.apache.catalina.core.StandardService - Starting service [Tomcat] 2020-06-13 08:25:29,289 [main] [INFO] org.apache.catalina.core.StandardEngine - Starting Servlet engine: [Apache Tomcat/9.0.21] 2020-06-13 08:25:29,304 [main] [INFO] org.apache.catalina.core.ContainerBase.[Tomcat-1].[localhost].[/] - Initializing Spring embedded WebApplicationContext 2020-06-13 08:25:29,305 [main] [INFO] org.springframework.web.context.ContextLoader - Root WebApplicationContext: initialization completed in 336 ms 2020-06-13 08:25:29,389 [main] [INFO] org.springframework.boot.actuate.endpoint.web.EndpointLinksResolver - Exposing 16 endpoint(s) beneath base path '/actuator' 2020-06-13 08:25:29,659 [main] [INFO] org.apache.coyote.http11.Http11NioProtocol - Starting ProtocolHandler ["http-nio-127.0.0.1-18082"] 2020-06-13 08:25:29,692 [main] [INFO] org.springframework.boot.web.embedded.tomcat.TomcatWebServer - Tomcat started on port(s): 18082 (http) with context path '' 2020-06-13 08:25:29,696 [main] [INFO] boot.Application - Started Application in 17.382 seconds (JVM running for 21.29)
上面的结果可以看到我刚刚发布的2个springboot的pod启动成功了，我们再看下其中一个pod的状态，执行如下命令
kubectl describe pod springboot-mybatis-deployment-9f5b78b76-2qtfg
[root@master k8s]# kubectl describe pod springboot-mybatis-deployment-9f5b78b76-2qtfg Name: springboot-mybatis-deployment-9f5b78b76-2qtfg Namespace: default Priority: 0 Node: worker1/192.168.59.138 Start Time: Sat, 13 Jun 2020 04:25:04 -0400 Labels: app=springboot-mybatis pod-template-hash=9f5b78b76 Annotations: <none> Status: Running IP: 10.244.1.2 IPs: IP: 10.244.1.2 Controlled By: ReplicaSet/springboot-mybatis-deployment-9f5b78b76 Containers: spingboot-mybatis: Container ID: docker://f3b0b7d1e2b9da0970e23134bbece26e4eb4fb0416e303119a0c4a34b954b666 Image: zjj2006forever/springboot-mybatis:1.2 Image ID: docker-pullable://zjj2006forever/springboot-mybatis@sha256:0e6f78256ab70ed5244042abc2a5bd5154a193fb9f97f05a3061e8334d387986 Port: 8300/TCP Host Port: 0/TCP State: Running Started: Sat, 13 Jun 2020 04:25:08 -0400 Ready: True Restart Count: 0 Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-cjp27 (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: default-token-cjp27: Type: Secret (a volume populated by a Secret) SecretName: default-token-cjp27 Optional: false QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s node.kubernetes.io/unreachable:NoExecute for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 3m53s default-scheduler Successfully assigned default/springboot-mybatis-deployment-9f5b78b76-2qtfg to worker1 Normal Pulled 3m50s kubelet, worker1 Container image "zjj2006forever/springboot-mybatis:1.2" already present on machine Normal Created 3m50s kubelet, worker1 Created container spingboot-mybatis Normal Started 3m49s kubelet, worker1 Started container spingboot-mybatis
上面的输出看到这个pod被调度到了worker1节点上，同时从上面的Events可以看到pod的启动过程。
3.测试
先让我们看一下我们刚刚部署的2个pod
root@master k8s]# kubectl get pod -l app -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES springboot-mybatis-deployment-5b78f66997-2hhdq 1/1 Running 0 7m23s 10.244.1.4 worker1 <none> <none> springboot-mybatis-deployment-5b78f66997-k87dx 1/1 Running 0 6m42s 10.244.1.5 worker1 <none> <none>
上面的输出结果中可以看到，这2个pod的ip地址是10.244.1.4和10.244.1.5，这是由flannel网络插件分配的。
接着我们进入容器内部看一下网络连通性
kubectl exec -it springboot-mybatis-deployment-5b78f66997-2hhdq -- /bin/sh
我们分别ping集群的主192.168.59.132，从节点192.168.59.138和mysql数据库地址192.168.59.1，发现都是正常返回数据包的
我们用curl看一下健康检查的地址，返回如下
[root@worker1 ~]# curl http://10.244.1.4:18082/actuator/health {"status":"UP","details":{"db":{"status":"UP","details":{"firstDataSource":{"status":"UP","details":{"database":"MySQL","hello":1}},"secondDataSource":{"status":"UP","details":{"database":"MySQL","hello":1}}}},"diskSpace":{"status":"UP","details":{"total":18238930944,"free":10361712640,"threshold":10485760}}}}
我们用浏览器输入地址http://10.244.1.4:8083/employee/jinjunzhu，返回505
这时在容器里面查看日志,输出见下面截图
cd /app/jar/logs/springboot-mybatis-deployment-5b78f66997-2hhdq/applog tail -f catalina.out
这个原因很明显，mysql没有对这个ip访问授权，执行一下授权命令：
//对ip地址192.168.59.132授权 grant all privileges on *.* to 'root'@'192.168.59.132' identified by '123456'; //是授权立即生效 flush privileges;
再次访问上面的url，成功，见下图
pod中的日志如下：
