k8s-部署jenkins

简介: jenkins部署jenkins和k8s集成jenkins基于k8s,执行pipline

部署jenkins


pvc配置


kind: PersistentVolumeClaim
apiVersion: v1
metadata:  name: jenkins-pvc
  namespace: jenkins-k8s
spec:  resources:    requests:      storage: 10Gi
  accessModes:    - ReadWriteMany
  storageClassName: nfs-csi


TIPS:

storageClassName: nfs-csi


采用的是nfs-csi,进行存储,需要安装 nfs-csi存储插件。

请参考  《k8s-存储(pv/pvc)》这篇文档



service配置



apiVersion: v1
kind: Service
metadata:  name: jenkins-service
  namespace: jenkins-k8s
  labels:    app: jenkins
spec:  selector:    app: jenkins
  type: ClusterIP
  ports:    - port: 8090      name: web
      targetPort: web
    - name: agent
      port: 50000      targetPort: agent



deploy配置


apiVersion: apps/v1
kind: Deployment
metadata:  name: jenkins
  namespace: jenkins-k8s
spec:  replicas: 1  selector:    matchLabels:      app: jenkins
  template:    metadata:      labels:        app: jenkins
    spec:      serviceAccountName: jenkins-k8s-sa
      containers:        - name: jenkins
          image: jenkins/jenkins:latest
          imagePullPolicy: IfNotPresent
          ports:            - containerPort: 8080              name: web
              protocol: TCP
            - containerPort: 50000              name: agent
              protocol: TCP
          resources:            limits:              cpu: 1000m
              memory: 1Gi
            requests:              cpu: 500m
              memory: 500Mi
          livenessProbe:            httpGet:              port: 8080              path: /login
            initialDelaySeconds: 60            timeoutSeconds: 5            failureThreshold: 12          readinessProbe:            httpGet:              port: 8080              path: /login
            initialDelaySeconds: 60          volumeMounts:            - mountPath: /var/jenkins_home
              name: jenkins-volume
              subPath: jenkins-home
#nodeSelector:#  app: jenkins      volumes:        - name: jenkins-volume
          persistentVolumeClaim:            claimName: jenkins-pvc


ingress配置


apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:  name: jk
  namespace: jenkins-k8s
spec:  ingressClassName: nginx
  rules:  - host: jk.tuling.com
    http:      paths:      - pathType: Prefix
        path: "/"        backend:          service:            name: jenkins-service
            port:              number: 8090


ingressClassName: nginx

使用的是ingress-nginx的负载均衡器,安装和使用请查看

k8s-负载均衡流量(ingress-nginx)》文章



安装jenkins



kubectl apply -f pvc.yaml -n jenkins-k8s
// 分别安装 ingress, service ,deploy
kubectl apply -f . -n jenkins-k8s  


查看结果

╰─# kubectl get pods 
NAME                       READY   STATUS    RESTARTS   AGE
curl                       1/1     Running   2          21h
jenkins-6c5f7cfd68-2njl7   1/1     Running   1          136m
╭─root@k201 ~/k8s/k8s-jenkins 
╰─#


╭─root@k201 ~/k8s/k8s-jenkins 
╰─# kubectl get ingress
NAME   CLASS   HOSTS           ADDRESS        PORTS   AGE
jk     nginx   jk.tuling.com   10.20.167.19   80      139m
╭─root@k201 ~/k8s/k8s-jenkins 


╭─root@k201 ~/k8s/k8s-jenkins 
╰─# kubectl get svc
NAME              TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)              AGE
jenkins-service   ClusterIP   10.20.56.146   <none>        8090/TCP,50000/TCP   140m
╭─root@k201 ~/k8s/k8s-jenkins 
╰─#


ingress访问jenkins


查看ingress的HOST 以及 ingress-controller 的 PORT

image.png


配置 /etc/hosts, 宿主机和host进行映射

image.png



访问地址: http://jenkins.test.com:32418/



jenkins集成k8s集群


首先登录jenkins控制台,首次登录需要输入初始化密码,查看方式


  1. 查看pvc目录pvc-d61907af-7904-4b43-b90d-c141ad072883
  2. 根据查询的pvc目录,进入此目录 pvc-d61907af-7904-4b43-b90d-c141ad072883
  3. 然后进入jenkins-home/secrets下,找到initialAdminPassword文件
  4. 查看initialAdminPassword文件
╭─root@k201 /nfs-server 
╰─# cd /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883 
╰─#
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883 
╰─#
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883 
╰─#
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883 
╰─# ls
jenkins-home
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883 
╰─# cd jenkins-home 
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883/jenkins-home 
╰─# ls
com.cloudbees.hudson.plugins.folder.config.AbstractFolderConfiguration.xml  jenkins.install.InstallUtil.lastExecVersion                              org.jenkinsci.plugins.github_branch_source.GitHubConfiguration.xml
config.xml                                                                  jenkins.install.UpgradeWizard.state                                      org.jenkinsci.plugins.workflow.flow.FlowExecutionList.xml
copy_reference_file.log                                                     jenkins.metrics.api.MetricsAccessKey.xml                                 org.jenkinsci.plugins.workflow.flow.GlobalDefaultFlowDurabilityLevel.xml      
github-plugin-configuration.xml                                             jenkins.model.ArtifactManagerConfiguration.xml                           org.jenkinsci.plugins.workflow.libs.GlobalLibraries.xml
hudson.model.UpdateCenter.xml                                               jenkins.model.GlobalBuildDiscarderConfiguration.xml                      plugins
hudson.plugins.build_timeout.global.GlobalTimeOutConfiguration.xml          jenkins.model.JenkinsLocationConfiguration.xml                           queue.xml
hudson.plugins.build_timeout.operations.BuildStepOperation.xml              jenkins.security.apitoken.ApiTokenPropertyConfiguration.xml              queue.xml.bak
hudson.plugins.emailext.ExtendedEmailPublisher.xml                          jenkins.security.QueueItemAuthenticatorConfiguration.xml                 scriptApproval.xml
hudson.plugins.git.GitSCM.xml                                               jenkins.security.ResourceDomainConfiguration.xml                         secret.key
hudson.plugins.git.GitTool.xml                                              jenkins.security.UpdateSiteWarningsConfiguration.xml                     secret.key.not-so-secret
hudson.plugins.timestamper.TimestamperConfig.xml                            jenkins.tasks.filters.EnvVarsFilterGlobalConfiguration.xml               secrets
hudson.tasks.Mailer.xml                                                     jenkins.telemetry.Correlator.xml                                         updates
hudson.tasks.Shell.xml                                                      jobs                                                                     userContent
hudson.triggers.SCMTrigger.xml                                              logs                                                                     users
identity.key.enc                                                            nodeMonitors.xml                                                         war
io.jenkins.plugins.junit.storage.JunitTestResultStorageConfiguration.xml    nodes
jenkins.fingerprints.GlobalFingerprintConfiguration.xml                     org.jenkinsci.plugins.gitclient.GitHostKeyVerificationConfiguration.xml
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883/jenkins-home 
╰─# cd secrets     
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883/jenkins-home/secrets 
╰─# ls
hudson.console.ConsoleNote.MAC  jenkins.model.Jenkins.crumbSalt                                    org.jenkinsci.plugins.workflow.log.ConsoleAnnotators.consoleAnnotator
hudson.model.Job.serverCookie   jenkins.slaves.JnlpSlaveAgentProtocol.secret                       org.springframework.security.web.authentication.rememberme.TokenBasedRememberMeServices.mac
hudson.util.Secret              master.key
initialAdminPassword            org.jenkinsci.main.modules.instance_identity.InstanceIdentity.KEY
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883/jenkins-home/secrets 
╰─# more initialAdminPassword 
1438047756bd41f69de930d5321c52f1
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883/jenkins-home/secrets 
╰─#


集成k8s插件


进入jenkins控制台之后,集成k8s插件,在插件管理,搜索kubernetes插件并安装


image.png



配置k8s云


进入 Dashboard -> 系统管理  -> 节点列表


image.png




image.png


名称:kubernetes

是在default空间下的service 名称


如图所示进行查看

image.png


kubernetes地址:https://kubernetes.default

此地址就是:kubernetes 服务名称 + 命名空间



Kubernetes 服务证书 key


查看/root/.kube/config文件中的certificate-authority-data,生成ca.crt文件

并查看ca.crt内容。


echo 'certificate-authority-data:值' | base64 -d > ca.crt


image.png



Kubernetes 服务证书 key,就是上面的输出的内容。





image.png


Kubernetes 命名空间,定义部署pod的命名空间



凭证 -> 添加


image.png



需要生成证书client.pfx文件



image.png

根据上面标注红色的key对应的值,生成不同文件。


相关命令:


echo 'certificate-authority-data:值' | base64 -d > ca.crt


echo 'client-certificate-data:值' | base64 -d > client.crt


echo 'client-key-data:值' | base64 -d > client.key


image.png



根据以上三个文件生成client.pfx文件



 openssl pkcs12 -export -out ./client.pfx -inkey ./client.key -in ./client.crt -certfile ./ca.crt

生成证书需要,输入密码,要记住密码。此密码就是在配置界面中的密码。


如:

image.png


证书配置完成以后,连接测试

如果出现 'Connected to Kubernetes v1.21.5' 说明提供的证书有效,就可以连接到k8s集群中了。


image.png

配置 jenkins地址和jenkins通道


jenkins地址:就是k8s集群中,jenkins的service 的地址。

http:// jenkins-service的host  + port


查看jenkins-service的host:


╰─# kubectl run curl --image=radial/busyboxplus:curl -it


进入 curl容器中,通过nslookup jenkins-service 的名称,查看 host

image.png

标注的就是 jenkins-service的host



端口查看:


image.png

标注的8090就是jenkins服务的端口


最终的jenkins的地址: http://jenkins-service.jenkins-k8s.svc.cluster.local:8090



jenkins的通道就是: jenkins-service.jenkins-k8s.svc.cluster.local:50000


这里的50000是,jenkins服务暴露的agent端口号

如图所示


image.png



创建一个job验证pipline


在主页,新建任务,test_slave

image.png


在任务的配置界面,配置流水线。输入一下内容


image.png

pipeline {  agent {    kubernetes {      yaml """apiVersion: v1
kind: Pod
metadata:  labels:    agent: test
spec:  containers:  - name: jnlp
    image: jenkins/jnlp-slave:4.13.2-1-jdk11
    imagePullPolicy: IfNotPresent
    tty: true"""}}  stages {    stage('Run maven') {      steps {        container('jnlp') {          sh 'java -version'}}}}}


这里的agent,是 jenkins/jnlp-slave:4.13.2-1-jdk11,这是镜像是jenkins官方提供的slave,基于 这个镜像,定义stages,并在当前容器,实行sh


TIPS

jenkins/jnlp-slave:4.13.2-1-jdk11这个镜像的jdk需要11, 由于 jenkins-service的的jdk是11,所以 jnlp-slave镜像的jdk需要是11,两种需要保持一致。


否则, k8s在调度jnlp-slave时,因jdk版本不一致,会报一下错误:


INFO: Connected
Aug 31, 2022 3:20:59 AM hudson.remoting.UserRequest perform
WARNING: LinkageError while performing UserRequest:hudson.slaves.SlaveComputer$SlaveVersion@2fd6ba1e
java.lang.UnsupportedClassVersionError: Failed to load hudson.slaves.SlaveComputer$SlaveVersion
        at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:416)
        at hudson.remoting.RemoteClassLoader.loadRemoteClass(RemoteClassLoader.java:256)
        at hudson.remoting.RemoteClassLoader.loadWithMultiClassLoader(RemoteClassLoader.java:225)
        at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:184)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at hudson.remoting.MultiClassLoaderSerializer$Input.resolveClass(MultiClassLoaderSerializer.java:132)
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1986)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1850)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2160)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1667)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:503)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:461)
        at hudson.remoting.UserRequest.deserialize(UserRequest.java:289)
        at hudson.remoting.UserRequest.perform(UserRequest.java:189)
        at hudson.remoting.UserRequest.perform(UserRequest.java:54)
        at hudson.remoting.Request$2.run(Request.java:376)
        at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.UnsupportedClassVersionError: hudson/slaves/SlaveComputer$SlaveVersion has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:635)
        at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:414)
        ... 24 more
Aug 31, 2022 3:20:59 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Terminated




运行pipline



image.png


查看运行结果:


image.png

输出内容:

Started by user admin
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline]{[Pipeline] node
Created Pod: kubernetes jenkins-k8s/test-slave-12-9v372-2tldh-9b0pm
Still waiting to schedule task
‘test-slave-12-9v372-2tldh-9b0pm’ is offline
Agent test-slave-12-9v372-2tldh-9b0pm is provisioned from template test_slave_12-9v372-2tldh
---apiVersion: "v1"kind: "Pod"metadata:  annotations:    buildUrl: "http://jenkins-service.jenkins-k8s.svc.cluster.local:8090/job/test_slave/12/"    runUrl: "job/test_slave/12/"  labels:    agent: "test"    jenkins: "slave"    jenkins/label-digest: "0e894596e8cbf0d5fa778a3e3ac32279186d4e0d"    jenkins/label: "test_slave_12-9v372"  name: "test-slave-12-9v372-2tldh-9b0pm"  namespace: "jenkins-k8s"spec:  containers:  - env:    - name: "JENKINS_SECRET"      value: "********"    - name: "JENKINS_TUNNEL"      value: "jenkins-service.jenkins-k8s.svc.cluster.local:50000"    - name: "JENKINS_AGENT_NAME"      value: "test-slave-12-9v372-2tldh-9b0pm"    - name: "JENKINS_NAME"      value: "test-slave-12-9v372-2tldh-9b0pm"    - name: "JENKINS_AGENT_WORKDIR"      value: "/home/jenkins/agent"    - name: "JENKINS_URL"      value: "http://jenkins-service.jenkins-k8s.svc.cluster.local:8090/"    image: "jenkins/jnlp-slave:4.13.2-1-jdk11"    imagePullPolicy: "IfNotPresent"    name: "jnlp"    resources:      limits: {}      requests:        memory: "256Mi"        cpu: "100m"    tty: true    volumeMounts:    - mountPath: "/home/jenkins/agent"      name: "workspace-volume"      readOnly: false  nodeSelector:    kubernetes.io/os: "linux"  restartPolicy: "Never"  volumes:  - emptyDir:      medium: ""    name: "workspace-volume"Running on test-slave-12-9v372-2tldh-9b0pm in /home/jenkins/agent/workspace/test_slave
[Pipeline]{[Pipeline] stage
[Pipeline]{ (Run maven)
[Pipeline] container
[Pipeline]{[Pipeline] sh
+ java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)
[Pipeline]}[Pipeline] // container
[Pipeline]}[Pipeline] // stage
[Pipeline]}[Pipeline] // node
[Pipeline]}[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: SUCCESS
相关实践学习
容器服务Serverless版ACK Serverless 快速入门:在线魔方应用部署和监控
通过本实验,您将了解到容器服务Serverless版ACK Serverless 的基本产品能力,即可以实现快速部署一个在线魔方应用,并借助阿里云容器服务成熟的产品生态,实现在线应用的企业级监控,提升应用稳定性。
云原生实践公开课
课程大纲 开篇:如何学习并实践云原生技术 基础篇: 5 步上手 Kubernetes 进阶篇:生产环境下的 K8s 实践 相关的阿里云产品:容器服务&nbsp;ACK 容器服务&nbsp;Kubernetes&nbsp;版(简称&nbsp;ACK)提供高性能可伸缩的容器应用管理能力,支持企业级容器化应用的全生命周期管理。整合阿里云虚拟化、存储、网络和安全能力,打造云端最佳容器化应用运行环境。 了解产品详情:&nbsp;https://www.aliyun.com/product/kubernetes
相关文章
|
15天前
|
Kubernetes 开发工具 Docker
微服务实践k8s与dapr开发部署实验(2)状态管理
微服务实践k8s与dapr开发部署实验(2)状态管理
36 3
微服务实践k8s与dapr开发部署实验(2)状态管理
|
29天前
|
Kubernetes 应用服务中间件 Docker
Kubernetes学习-集群搭建篇(二) 部署Node服务,启动JNI网络插件
Kubernetes学习-集群搭建篇(二) 部署Node服务,启动JNI网络插件
|
2天前
|
Kubernetes 数据处理 调度
天呐!部署 Kubernetes 模式的 Havenask 集群太震撼了!
【6月更文挑战第11天】Kubernetes 与 Havenask 集群结合,打造高效智能的数据处理解决方案。Kubernetes 如指挥家精准调度资源,Havenask 快速响应查询,简化复杂任务,优化资源管理。通过搭建 Kubernetes 环境并配置 Havenask,实现高可扩展性和容错性,保障服务连续性。开发者因此能专注业务逻辑,享受自动化基础设施管理带来的便利。这项创新技术组合引领未来,开启数据处理新篇章。拥抱技术新时代!
|
6天前
|
Cloud Native jenkins Java
使用Jenkins实现持续集成与持续部署
【6月更文挑战第7天】本文介绍了如何使用Jenkins实现持续集成与持续部署,提高软件开发效率和质量。首先,解释了CI/CD的概念,持续集成通过自动化构建和测试减少错误,持续部署则自动将软件部署至生产环境。接着,详细阐述了Jenkins的安装配置、构建项目设置,以及如何通过代码提交触发构建、自动化测试和构建报告。此外,还讨论了Jenkins的持续部署功能,包括配置部署环境、自动化部署和回滚策略。最后,指出Jenkins在DevOps和云原生趋势中的重要角色。
|
9天前
|
Kubernetes 容器 Perl
k8s部署seata 报错 没有提供足够的身份验证信息 [ http-nio-7091-exec-2] [ty.JwtAuthenticationEntryPoint] [ commence] [] : Responding with unauthorized error. Message - Full authentication is required to access this resource
Kubernetes pod 在16:12时出现两次错误,错误信息显示需要完整认证才能访问资源。尽管有此错误,但页面可正常访问。附有yaml配置文件的图片。
21 2
|
12天前
|
Kubernetes 负载均衡 应用服务中间件
k8s 二进制安装 优化架构之 部署负载均衡,加入master02
k8s 二进制安装 优化架构之 部署负载均衡,加入master02
|
14天前
|
jenkins 持续交付 开发工具
蓝易云 - 基于Jenkins自动打包并部署docker环境
以上就是基于Jenkins自动打包并部署docker环境的基本步骤。在实际操作中,你可能需要根据你的具体需求进行一些调整。
124 0
|
15天前
|
Kubernetes Docker 微服务
微服务实践k8s&dapr开发部署实验(3)订阅发布
微服务实践k8s&dapr开发部署实验(3)订阅发布
29 0
|
26天前
|
前端开发 JavaScript 安全
Mac部署Jenkins(保姆级教程!)_前端jenkins自动部署 mac
Mac部署Jenkins(保姆级教程!)_前端jenkins自动部署 mac
|
27天前
|
Kubernetes 前端开发 容器
k8s部署模板
k8s部署模板

推荐镜像

更多