著作权归作者所有，任何形式的转载都请联系作者获得授权并注明出处。
docker swarm简介
Docker Swarm 和 Docker Compose 一样，都是 Docker 官方容器编排项目，但不同的是，Docker Compose 是一个在单个服务器或主机上创建多个容器的工具，而 Docker Swarm 则可以在多个服务器或主机上创建容器集群服务，对于微服务的部署，显然 Docker Swarm 会更加适合。
从 Docker 1.12.0 版本开始，Docker Swarm 已经包含在 Docker 引擎中（docker swarm），并且已经内置了服务发现工具，我们就不需要像之前一样，再配置 Etcd 或者 Consul 来进行服务发现配置了。
docker swarm 体系结构如如所示：
一个Manager下面有多个Worker（实际运行中每个都是一个container）
下图是一个Service和Replicas（复制品）模型图， service是nginx，但是下面有3个replicas nginx构成了一个集群
如何创建swarm集群
swarm node节点的分配是默认随机，起初我们是不知道的
docker swarm --help #查看帮助 docker swarm init --advertise-addr=198.168.205.10 #初始化一个docker swarm manager docker node ls
- 第一台机器，执行命令设置为主节点
[vagrant@swarm-manager~]$docker swarminit--advertise-addr=192.168.205.10 Swarm initialized:current node (ywpy3eosncgw413bervl9kgc7)is now manager To add a worker to this swarm, run the following command: I docker swarmjoin--tokenSWMTKN-1-3cv8y6rb944uc6j2bub73jjvdue7vo7p8hr38005lj5kf290g3-6zlwprjtwrxkjn90jua5l4q 192.168.205.10:2377 Toadd a manager to this swarm, run 'docker swarm join-token manager'and follow the instrucyions
- 第二台机器，执行上图中倒数第二段的命令
[ vagrant @ swarm -worker1~]$ docker swarm join -- token SWMTKN -1-3cv8y6rb944uc6j2bub73jjvdue7vo7p8hr380o5lj5k t290g5-bzLwprtwrxkmx5g0ua5L4q192.168.205.10:25 This node joined a swarm as a worker 。
- 然后回到第一台manager机器查看一下当前节点
[vagrant@swarm-manager docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS hahlhop9aouynv5893nrw9moj ywpy3eosncgw413bervl9kgc7* Iswarm-manager swarm-worker1 Ready Ready Active Active
- 剩下的几台机器依次执行同样的命令
[ vagrant @ swarm -worker1~]$ docker swarm join -- token SWMTKN -1-3cv8y6rb944uc6j2bub73jjvdue7vo7p8hr380o5lj5k t290g5-bzLwprtwrxkmx5g0ua5L4q192.168.205.10:25 This node joined a swarm as a worker
- 再次查看第一台manager机器，会发现已经有3个docker swarm node
[vagrant@swarm-manager ~]$ docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ywpy3eosncgw413bervl9kgc7 * swarm-manager Ready Active Leader hahlhop9aouynv5893nrw9moj Le3hfq08q3r01uz4h6ghb570m swarm-worker1 swarm-worker2 Ready Ready Active Active
- 至此，一个简单的docker swarm集群创建完成
创建维护和水平扩展
docker service create --help docker service ls #查看所有 docker service scale #水平扩展 docker service scale [serviceName] = [num] #设置service（container）数目 docker service ps [serviceNmae] #查看当前service详细 docker ps
- 举例： 执行命令
docker service create --name demo busybox sh -c"while true;do sleep 3600;done"
[nodel] (local) root@192.168.0.18 ~ $docker service create --name demo busybox sh -c "while true;do sleep 3600;done' znrtsx3w2ekppkmgmdihhy9sn overall progress: 1 out of 1 tasks 1/1: running [- > verify: Service converged
- 执行命令
docker service ls查看，REPLICAS表示水平扩展 
- 执行命令
docker service ps demoNODE显示node3表示demo service在我们的第三台机器上 
- 执行命令
docker service scale=5，水平扩展5个相同的服务，此时REPLICAS是5/5，5/5是什么意思呢，第一个5是表示有多少个是ready（准备），第二个5是有多少个scheuld(分片实例 
- 执行命令
docker service ps demo可以看到该servie所有部署情况 
第一台机器，也就是Docker Swarm Manange执行docker service rm demo命令后，则以上创建的service demo 的container全部销毁，可以执行docker service ps demo, docker ps再次查看详情。
实战-swram集群里部署wordpress
docker network create -d overlay demo #创建名为demo的docker network docker network ls #查看 docker exec [containerID] ip a #查看容器ip
- 执行命令
docker service create --name mysql --env MYSQL_ROOT_PASSWORD=root --env MYSQL_DATABASE=wordpress --network demo --mount type=volume,source=mysql-data,destination=/var/lib/mysql mysq
[vagrant@swarm-manager~]$docker service create --name mysql --env MYSQL_ROOT_PASSWORD=root --env MYSQL_DATABASE=wordpress--networkdemo--mount type=volume,source=mysql-data,destination=/var/l b/mysql mysql snzj39jxw044gojs0evj1yxko overall progress: 1 out of 1 tasks 1/1:running verify: Service converged
- 查看service mysql部署情况
[vagrant@swarm-manager ~]$ docker service ps mysql ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 552puk51lz81 mysql.1 mysql:latest swarm-manager Running Running 23 seconds ago [vagrant@swarm-manager ~]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES eb85f0656e45 mysql:latest "docker-entrypoint.s.." 34 seconds ago 3306/tcp mysql.1.552puk51lz81f90xl9pi2we73
- 3.执行命令
docker service create --name wordpress -p 80:80 --env WORDPRESS_DB_PASSWORD=root --env WORDPRESS_DB_HOST=mysql --network demo wordpress
[vagrant@swarm-manager ~]$docker service create --name wordpress -p80:80 --env WORDPRESS_DB_PASS WORD=root --envWORDPRESS_DB_HOST=mysql --network demo wordpress utzivgyq796am70znise0eaqj overall progress: 1 out of 1 tasks1/1:running verify: Service converged [vagrant@swarm-manager ~]$ docker service ps wordpress ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 10uf3rlu51ju wordpress.1 wordpress:latest swarm-worker2 Rya Running 54 seconds ago
- 执行命令
docker service ps wordpress，docker ps查看wordpress部署情况
[vagrant@swarm-worker2 ~]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ddeacd221551 wordpress:latest "docker-entrypoint.s..." About a minute ago Up Abou ut a minute 80/tcp wordpress.1.10uf3rlu51jujqpfcv4qvsogm [vagrant@swarm-worker2 ~]$
swarm 底层机制会同步网络的创建，因为它要实现多个节点之间容器的通信。
swar
m集群服务间通信
docker network create -d overlay demo创建名为demo的docker network- 执行命令
docker service create --name whoami -p 8000:8000 --network demo -d jwilder/whoami
[vagrant@swarm-manager~]$docker service create --name whoami -p 8000:8000 --network demo -d jwilder/whoami image jwilder/whoami:latest could not be accessed on a registry to record its digest. Each node will access jwilder/whoami:latest independently, possibly leading to different nodes running different versions of the image.
- 执行命令
docker service ls，docker service ps whoami查看，可以看出service在当前机器上
[ vagranteswarm - manager ~]$ docker service Ls ID NAME MODE PORTS WhOaML REPLICAS IMAGE jwilde e1jr9b1e5l3z replicated r / whoami : latest *:8000->8000/tcp [ vagrant @ swarm - manager ~]$ docker service ps whoami D NAME SIRED STATE CURRENT STATE cen8tudtj8n2 whoami .1 nning Running 13 seconds qgo [ vagrant CONTAINER ID ATUS 5b792811ea3e 20 seconds 1/1 IMAGE ERROR jwilder / whoami : latest NODE P0RTS swarm - manager DE Ru :@ swarm - manager ~]$ docker pS AE ST COMMAND CREATED 21sec0nd PORTS jwilder / whoami : latest "/ app / http ” 8000/tCp_ NAMES whoami .1.cen8tudtj8n296csvgfn9yymp-
- 执行命令
curl 127.0.0.1:8000，结果返回了host name
[ vagrant @ swarm - manager ~]$ curl 127.0.0.1:8000 I ' m 5b792811ea3e
- 执行命令
docker service create --name client -d --network demo busybox sh -c "while true;do sleep 4600;done"
[ vagrant @ swarm - manager ~]$ docker service create -- name client - d -- network demo busyb ox sh - c " while true ; do sleep 3600; done " image busybox : latest could not be accessed on a registry to record its digest . Each node will access busybox : latest independently , possibly leading to different nodes running different Verstons otthe Lmage 。 303bqxlblaeaq2y83nbpja2ed
- 执行命令
docker service ls，docker service ps client查看，可以看出service在另一台机器上
Lvagrant @ swarm - manager ~]$ docker service ps client TD D STATE injkoccl08be NAME CURRENT STATE c1ient.1 Running 49 seconds ago IMAGE ERROR busybox : latest N0DE P0RTS swarm -worker1 DEsIRE Runnin
- 查看并进入service client
[vagrante swarm-worker1 docker ps CONTAINER 1D IMAGE COMMAND CREATED STATUS PORTS NAMES f9f80eb1dcf5 busybox:latest "sh -c 'while true; …" About a minute ago Up About a minute client.1.inikoccl08besw6t4woc3z5hs [vagrant@swarm-worker1 ~ $ docker exec-it f9f8 sh #
- 看到可以ping通whoami主机地址，但是主机地址显示10.0.0.7，很显然这不是真是的ip地址
/# ping whoami PING whoami (10.0.0.7) 56 data bytes 64 bytes fron 10.0.0.7: seq=0 ttl=64 time=0.043 ms 64 bytes fron 10.0.0.7: seq=1 ttl=64 time=0.127 ms 64 bytes fron 10.0.0.7: seq=2 ttl=64 time=0.087 ms 64 bytes fron 10.0.0.7: seq=3 ttl=64 time=0.088 ms ^C --- whoami ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max=0.043/0.086/0.127 ms / #
- 进入service whoami机器执行
docker service scale whoami=2，横向扩展2个
[vagrant@swarm-manager ~]$docker service scale whoami=2 whoami scaled to 2 overall progress: 2 out of 2 tasks1/2:running2/2:running verify: Service converged [vagrant@swarm-manager ~]$ docker service ps whoami ID SIRED STATE NAME CURRENT STATE IMAGE ERROR PORTS NODE DE cen8tudtj8n2 whoami.1 jwilder/whoami:latest swarm-manager Ru nning Running 16 minutes ago i1c0ys6wm1ak whoami.2 jwilder/whoami:latest swarm-worker2 Ru nning Running 39 seconds ago
- 再次重复第7、8步，还是同样的结果，其实这个一个vip（虚拟IP），可以使用
nslookup命令查看DNS
/# nslookup whoami 127.0.0.11 SerVer : Address 1:127.0.0.11 Name : whoami Address 1:10.0.0.7 # DNS 服务器地址 虚拟 IP
/# nslookup tasks . whoami Server : 1z7.0.0.11 Address 1:127.0.0.11 Name : Address 1:10.0.0.8 whoami .1.cen8tudtj8n296csvgfn9yymp. demo Address 2:10.0.0.16 whoami .2.i1c0ys6wm1ak067i68vgq3qwji. demo . tasks . whoami 真实容器 IP
- 在部署了service whoami的机器上执行
docker exec [containerID] ip a查看对应的容器IP地址，即可知道真实IP，VIP（虚拟IP）不会经常变动，机器IP可能变动
此时多次执行命令curl 127.0.0.1:8000，会发现每次返回的host name可能会变，这是因为已经提供了负载均衡功能了。
Internal Load Balancing 图：
DNS+VIP+iptables+LVS
Routing Mesh的两种体现
- Internal——Container和Container之间的访问通过overlay网络（通过VIP虚拟ip）
- Ingress——如果服务有绑定接口，则此服务可以通过任意swarm节点的相应接口访问
上述的操作分别采用了这2种方式
Ingress负载均衡
- 外部访问的负载均衡
- 服务端口被暴露到各个swarm节点
- 内部通过IPVS进行负载均衡
- 使用之前的例子service whoami，NODE分别在swarm-manager、swarm-manager2两台机器上
- 可以发现每次都会返回不同的host name，负载均衡
[ vagrant @ swarm - manager ~]$ curl 127.0.0.1:8000 I ' m d9b75ff1a594 [ vagrant @ swarm - manager ~]$ curl 127.0.0.1:8000 Im 5b792811ea3e [ vagrant @ swarm - manager ~]$ curl 127.0.0.1:8000 I ' m d9b75ff1a594 Lvagrant @ swarm - manager ~]$ curl 127.0.0.1:8000 I ' m 5b792811ea3e [ vagrant @ swarm - manager ~]$ curl 127.0.0.1:8000 I ' m d9b75ff1a594 Lvagrant @ swarm - manager ~]$ curl 127.0.0.1:8000 I ' m 5b792811ea3e
- swarm-manager1没有service whoami，也能访问
[ vagrant @ swarm -worker1~]$ curl 127.0.0.1:8000 I ' m d9b75ff1a594 [ vagrantc ]$ curl 127.0.0.1:8000 I ' m 5b792811ea3e
swarm-manager1执行sudo iptables -nL -t nat查看转发规则
Chain DOCKER-INGRESS(2 references) target prot opt source destination DNAT tcp 0.0.0.0/0 0.0.0.0/0 tcp dpt:8000 to:172.18.0 .2:8000 RETURN all -- 0.0.0.0/0 0.0.0.0/0 [vagrant@swarm-worker1 ~]$curl 127.0.0.1:8000 I'm d9b75ff1a594
执行ip a可以看出在同一网段
4:docker0:<NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN link/ether 02:42:e6:8d:34:e5 brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 docker_awbridge: <BROADCAST.MULTICAST,UP.LOWER_UP> mtu 1500 qdisc noqueue state Up 9 link/ether 02:42:c2:83:19:f4 brd ff:ff:ff:ff:ff:f inet 172.18.0.1/16 brd 172.18.255.255 scope globa docker_gwbridge valid_lft forever preferred_lft forever inet6 fe80::42:c2ff:fe83:19f4/64 scope link yUllu rt torever orerereu Llt ore 11:vethec89e00@if10:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP link/ether 9e:c5:26:52:0f:44 brd ff:ff:ff:ff:ff:ff link-netnsid 1 inet6 fe80::9cc5:26ff:fe52:f44/64 scope link
2. brctl show
[vagrant@swarm-worker1 ~]$ brctl show bridge name bridge id STP enabled interfaces docker0 8000.0242e68d34e5 no veth975c56e docker_gwbridge 8000.0242c28319f4 no vethec89e00
3. docker network ls docker network inspect docker_gwbridge
[vagrant@swarm-worker1 ~]$ docker network ls NETWORK ID NAME DRIVER SCOPE ef2d10b71ed5 bridge bridge local swarm heweupwlcull demo overlay 3bc9a28069c4 docker_gwbridge bridge local 0c5059ff84b3 host host local obe9f2751sv7 50ca943d108d none ingress null overlay local swarm [vagrant@swarm-worker1 ~]$docker network inspect docker_qwbriage
4. docker_gwbridge 具体内容
"ConfigOnly": false,"Containers":{ "f9f80eb1dcf5888efc3f1050bf04ef937e0af63b738773ef8e1b027128e36c79":{ "Name": "gateway_f9f80eb1dcf5"， Endpotrro::"e4e12bd68d9b1a4df42099063913a20f47f835d6a95335c9d0cff04 e49bd2571", "MacAddress": "02:42:ac:12:00:03","IPv4Address": "172.18.0.3/16"，"IPv6Address":""} ， "ingress-sbox":{ "Name": "gateway_ingress-sbox", "EndpointID":"11e289691864c53e2ddbd49f1e89d86c7cd7d5829225eb306266b31 a7c50d9a8", "02 60-场2# "IPv4Address": "172.18.0.2/16" IPv6Address": } ， "Options":{
5. sudo ls /var/run/docker/netns sudo nsenter --net=/var/run/docker/netns/ingress_sbox
[vagrant@swarm-worker1 ~]$sudo ls /var/run/docker/netns10de222a0ebc 1-heweupwlcu 1-obe9f2751s inaress sbox [vagrant@swarm-worker1~$nsenter --net=/var/run/docker/netns/ingress_sbox nsenter:cannot open/var/run/docker/netns/ingress_sbox: Permission denied [vagrant@swarm-worker1~]$sudonsenter--net=/var/run/docker/netns/ingress_sbox root@swarm-worker1 vagrant# ip a LO:<LUUPBACK,UP,LUWERUP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid lft forever breferred_lft forever 7:eth0@if8:<BROADCAST.MULTICAST.UP.LOWER_UP> mtu 1450 adisc noaueue state UP link/ether 02:42:0a:ff:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.255.0.3/16 brd 10.255.255.255 scope global eth0 valid lft forever preferred_lft forever 10:eth1@if11:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP link/ether02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 1 inet 172.18.0.2/16 brd 172.18.255.255 scope global eth1 valid lft forever preferred lft forever [root@swarm-worker1 vagrant]#
6. iptables -nL -t mangle，白色处具体是做负载均衡的
[root @ swarm -worker1 vagrant ]# iptables - nL - t mangle Chain PREROUTING ( policy ACCEPT ) target prot opt source MARK tcp --0.0.0.0/0 l01 destination 0.0.0.0/0 tCp dpt :8000 MARK set 0x Chain TNPUT ( policy ACCEPT ) target prot opt source destination Chain FORWARD ( policy ACcEPT ) target prot opt sourCe destination Chain ouTPuT ( policy AccepT ) target prot opt source MARK nTT destination 102s5 a 7 IMARK cet 0x101
7. 下载LVS管理工具，yum install ipvsadm8. 再次进入root，执行ipvsadm -l
Chain POSTROUTING (policy ACCEPT) target prot opt source destination Iroot@swarm-worker1 vagrant]# ipvsadm -l IP Virtual Server version 1.2.1(size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn FWM 257 rr 真实service地址 10.255.0.8:0 Masq 1 10.255.0.9:0 Masq 1 0 0 [root@swarm-worker1 vagrant]#
Ingress Network的数据包走向详情
