部署jenkins

pvc配置

kind: PersistentVolumeClaim
apiVersion: v1
metadata:  name: jenkins-pvc
  namespace: jenkins-k8s
spec:  resources:    requests:      storage: 10Gi
  accessModes:    - ReadWriteMany
  storageClassName: nfs-csi

TIPS:

storageClassName: nfs-csi

采用的是nfs-csi,进行存储，需要安装 nfs-csi存储插件。

请参考《k8s-存储(pv/pvc)》这篇文档

service配置

apiVersion: v1
kind: Service
metadata:  name: jenkins-service
  namespace: jenkins-k8s
  labels:    app: jenkins
spec:  selector:    app: jenkins
  type: ClusterIP
  ports:    - port: 8090      name: web
      targetPort: web
    - name: agent
      port: 50000      targetPort: agent

deploy配置

apiVersion: apps/v1
kind: Deployment
metadata:  name: jenkins
  namespace: jenkins-k8s
spec:  replicas: 1  selector:    matchLabels:      app: jenkins
  template:    metadata:      labels:        app: jenkins
    spec:      serviceAccountName: jenkins-k8s-sa
      containers:        - name: jenkins
          image: jenkins/jenkins:latest
          imagePullPolicy: IfNotPresent
          ports:            - containerPort: 8080              name: web
              protocol: TCP
            - containerPort: 50000              name: agent
              protocol: TCP
          resources:            limits:              cpu: 1000m
              memory: 1Gi
            requests:              cpu: 500m
              memory: 500Mi
          livenessProbe:            httpGet:              port: 8080              path: /login
            initialDelaySeconds: 60            timeoutSeconds: 5            failureThreshold: 12          readinessProbe:            httpGet:              port: 8080              path: /login
            initialDelaySeconds: 60          volumeMounts:            - mountPath: /var/jenkins_home
              name: jenkins-volume
              subPath: jenkins-home
#nodeSelector:#  app: jenkins      volumes:        - name: jenkins-volume
          persistentVolumeClaim:            claimName: jenkins-pvc

ingress配置

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:  name: jk
  namespace: jenkins-k8s
spec:  ingressClassName: nginx
  rules:  - host: jk.tuling.com
    http:      paths:      - pathType: Prefix
        path: "/"        backend:          service:            name: jenkins-service
            port:              number: 8090

ingressClassName: nginx

使用的是ingress-nginx的负载均衡器，安装和使用请查看

《k8s-负载均衡流量(ingress-nginx)》文章

安装jenkins

kubectl apply -f pvc.yaml -n jenkins-k8s
// 分别安装 ingress， service ，deploy
kubectl apply -f . -n jenkins-k8s

查看结果

╰─# kubectl get pods 
NAME                       READY   STATUS    RESTARTS   AGE
curl                       1/1     Running   2          21h
jenkins-6c5f7cfd68-2njl7   1/1     Running   1          136m
╭─root@k201 ~/k8s/k8s-jenkins 
╰─#

╭─root@k201 ~/k8s/k8s-jenkins 
╰─# kubectl get ingress
NAME   CLASS   HOSTS           ADDRESS        PORTS   AGE
jk     nginx   jk.tuling.com   10.20.167.19   80      139m
╭─root@k201 ~/k8s/k8s-jenkins

╭─root@k201 ~/k8s/k8s-jenkins 
╰─# kubectl get svc
NAME              TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)              AGE
jenkins-service   ClusterIP   10.20.56.146   <none>        8090/TCP,50000/TCP   140m
╭─root@k201 ~/k8s/k8s-jenkins 
╰─#

ingress访问jenkins

查看ingress的HOST 以及 ingress-controller 的 PORT

配置 /etc/hosts, 宿主机和host进行映射

访问地址： http://jenkins.test.com:32418/

jenkins集成k8s集群

首先登录jenkins控制台，首次登录需要输入初始化密码，查看方式

查看pvc目录pvc-d61907af-7904-4b43-b90d-c141ad072883
根据查询的pvc目录，进入此目录 pvc-d61907af-7904-4b43-b90d-c141ad072883
然后进入jenkins-home/secrets下，找到initialAdminPassword文件
查看initialAdminPassword文件

╭─root@k201 /nfs-server 
╰─# cd /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883 
╰─#
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883 
╰─#
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883 
╰─#
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883 
╰─# ls
jenkins-home
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883 
╰─# cd jenkins-home 
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883/jenkins-home 
╰─# ls
com.cloudbees.hudson.plugins.folder.config.AbstractFolderConfiguration.xml  jenkins.install.InstallUtil.lastExecVersion                              org.jenkinsci.plugins.github_branch_source.GitHubConfiguration.xml
config.xml                                                                  jenkins.install.UpgradeWizard.state                                      org.jenkinsci.plugins.workflow.flow.FlowExecutionList.xml
copy_reference_file.log                                                     jenkins.metrics.api.MetricsAccessKey.xml                                 org.jenkinsci.plugins.workflow.flow.GlobalDefaultFlowDurabilityLevel.xml      
github-plugin-configuration.xml                                             jenkins.model.ArtifactManagerConfiguration.xml                           org.jenkinsci.plugins.workflow.libs.GlobalLibraries.xml
hudson.model.UpdateCenter.xml                                               jenkins.model.GlobalBuildDiscarderConfiguration.xml                      plugins
hudson.plugins.build_timeout.global.GlobalTimeOutConfiguration.xml          jenkins.model.JenkinsLocationConfiguration.xml                           queue.xml
hudson.plugins.build_timeout.operations.BuildStepOperation.xml              jenkins.security.apitoken.ApiTokenPropertyConfiguration.xml              queue.xml.bak
hudson.plugins.emailext.ExtendedEmailPublisher.xml                          jenkins.security.QueueItemAuthenticatorConfiguration.xml                 scriptApproval.xml
hudson.plugins.git.GitSCM.xml                                               jenkins.security.ResourceDomainConfiguration.xml                         secret.key
hudson.plugins.git.GitTool.xml                                              jenkins.security.UpdateSiteWarningsConfiguration.xml                     secret.key.not-so-secret
hudson.plugins.timestamper.TimestamperConfig.xml                            jenkins.tasks.filters.EnvVarsFilterGlobalConfiguration.xml               secrets
hudson.tasks.Mailer.xml                                                     jenkins.telemetry.Correlator.xml                                         updates
hudson.tasks.Shell.xml                                                      jobs                                                                     userContent
hudson.triggers.SCMTrigger.xml                                              logs                                                                     users
identity.key.enc                                                            nodeMonitors.xml                                                         war
io.jenkins.plugins.junit.storage.JunitTestResultStorageConfiguration.xml    nodes
jenkins.fingerprints.GlobalFingerprintConfiguration.xml                     org.jenkinsci.plugins.gitclient.GitHostKeyVerificationConfiguration.xml
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883/jenkins-home 
╰─# cd secrets     
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883/jenkins-home/secrets 
╰─# ls
hudson.console.ConsoleNote.MAC  jenkins.model.Jenkins.crumbSalt                                    org.jenkinsci.plugins.workflow.log.ConsoleAnnotators.consoleAnnotator
hudson.model.Job.serverCookie   jenkins.slaves.JnlpSlaveAgentProtocol.secret                       org.springframework.security.web.authentication.rememberme.TokenBasedRememberMeServices.mac
hudson.util.Secret              master.key
initialAdminPassword            org.jenkinsci.main.modules.instance_identity.InstanceIdentity.KEY
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883/jenkins-home/secrets 
╰─# more initialAdminPassword 
1438047756bd41f69de930d5321c52f1
╭─root@k201 /nfs-server/pvc-d61907af-7904-4b43-b90d-c141ad072883/jenkins-home/secrets 
╰─#

集成k8s插件

进入jenkins控制台之后，集成k8s插件，在插件管理，搜索kubernetes插件并安装

配置k8s云

进入 Dashboard -> 系统管理 -> 节点列表

名称：kubernetes

是在default空间下的service 名称

如图所示进行查看

kubernetes地址：https://kubernetes.default

此地址就是：kubernetes 服务名称 + 命名空间

Kubernetes 服务证书 key

查看/root/.kube/config文件中的certificate-authority-data，生成ca.crt文件

并查看ca.crt内容。

echo 'certificate-authority-data:值' | base64 -d > ca.crt

Kubernetes 服务证书 key，就是上面的输出的内容。

Kubernetes 命名空间，定义部署pod的命名空间

凭证 -> 添加

需要生成证书client.pfx文件

根据上面标注红色的key对应的值，生成不同文件。

创建一个job验证pipline

在主页，新建任务，test_slave

在任务的配置界面，配置流水线。输入一下内容

pipeline {  agent {    kubernetes {      yaml """apiVersion: v1
kind: Pod
metadata:  labels:    agent: test
spec:  containers:  - name: jnlp
    image: jenkins/jnlp-slave:4.13.2-1-jdk11
    imagePullPolicy: IfNotPresent
    tty: true"""}}  stages {    stage('Run maven') {      steps {        container('jnlp') {          sh 'java -version'}}}}}

这里的agent，是 jenkins/jnlp-slave:4.13.2-1-jdk11，这是镜像是jenkins官方提供的slave，基于这个镜像，定义stages，并在当前容器，实行sh

TIPS

jenkins/jnlp-slave:4.13.2-1-jdk11这个镜像的jdk需要11，由于 jenkins-service的的jdk是11，所以 jnlp-slave镜像的jdk需要是11，两种需要保持一致。

否则， k8s在调度jnlp-slave时，因jdk版本不一致，会报一下错误：

INFO: Connected
Aug 31, 2022 3:20:59 AM hudson.remoting.UserRequest perform
WARNING: LinkageError while performing UserRequest:hudson.slaves.SlaveComputer$SlaveVersion@2fd6ba1e
java.lang.UnsupportedClassVersionError: Failed to load hudson.slaves.SlaveComputer$SlaveVersion
        at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:416)
        at hudson.remoting.RemoteClassLoader.loadRemoteClass(RemoteClassLoader.java:256)
        at hudson.remoting.RemoteClassLoader.loadWithMultiClassLoader(RemoteClassLoader.java:225)
        at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:184)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at hudson.remoting.MultiClassLoaderSerializer$Input.resolveClass(MultiClassLoaderSerializer.java:132)
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1986)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1850)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2160)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1667)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:503)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:461)
        at hudson.remoting.UserRequest.deserialize(UserRequest.java:289)
        at hudson.remoting.UserRequest.perform(UserRequest.java:189)
        at hudson.remoting.UserRequest.perform(UserRequest.java:54)
        at hudson.remoting.Request$2.run(Request.java:376)
        at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.UnsupportedClassVersionError: hudson/slaves/SlaveComputer$SlaveVersion has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:635)
        at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:414)
        ... 24 more
Aug 31, 2022 3:20:59 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Terminated

运行pipline

查看运行结果：

输出内容：

Started by user admin
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline]{[Pipeline] node
Created Pod: kubernetes jenkins-k8s/test-slave-12-9v372-2tldh-9b0pm
Still waiting to schedule task
‘test-slave-12-9v372-2tldh-9b0pm’ is offline
Agent test-slave-12-9v372-2tldh-9b0pm is provisioned from template test_slave_12-9v372-2tldh
---apiVersion: "v1"kind: "Pod"metadata:  annotations:    buildUrl: "http://jenkins-service.jenkins-k8s.svc.cluster.local:8090/job/test_slave/12/"    runUrl: "job/test_slave/12/"  labels:    agent: "test"    jenkins: "slave"    jenkins/label-digest: "0e894596e8cbf0d5fa778a3e3ac32279186d4e0d"    jenkins/label: "test_slave_12-9v372"  name: "test-slave-12-9v372-2tldh-9b0pm"  namespace: "jenkins-k8s"spec:  containers:  - env:    - name: "JENKINS_SECRET"      value: "********"    - name: "JENKINS_TUNNEL"      value: "jenkins-service.jenkins-k8s.svc.cluster.local:50000"    - name: "JENKINS_AGENT_NAME"      value: "test-slave-12-9v372-2tldh-9b0pm"    - name: "JENKINS_NAME"      value: "test-slave-12-9v372-2tldh-9b0pm"    - name: "JENKINS_AGENT_WORKDIR"      value: "/home/jenkins/agent"    - name: "JENKINS_URL"      value: "http://jenkins-service.jenkins-k8s.svc.cluster.local:8090/"    image: "jenkins/jnlp-slave:4.13.2-1-jdk11"    imagePullPolicy: "IfNotPresent"    name: "jnlp"    resources:      limits: {}      requests:        memory: "256Mi"        cpu: "100m"    tty: true    volumeMounts:    - mountPath: "/home/jenkins/agent"      name: "workspace-volume"      readOnly: false  nodeSelector:    kubernetes.io/os: "linux"  restartPolicy: "Never"  volumes:  - emptyDir:      medium: ""    name: "workspace-volume"Running on test-slave-12-9v372-2tldh-9b0pm in /home/jenkins/agent/workspace/test_slave
[Pipeline]{[Pipeline] stage
[Pipeline]{ (Run maven)
[Pipeline] container
[Pipeline]{[Pipeline] sh
+ java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)
[Pipeline]}[Pipeline] // container
[Pipeline]}[Pipeline] // stage
[Pipeline]}[Pipeline] // node
[Pipeline]}[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: SUCCESS

k8s-部署jenkins

部署jenkins

pvc配置

service配置

deploy配置

ingress配置

安装jenkins

查看结果

ingress访问jenkins

jenkins集成k8s集群

集成k8s插件

配置k8s云

创建一个job验证pipline

运行pipline

热门文章

最新文章

相关课程

相关电子书

相关实验场景

推荐镜像