用saltstack cp模块实现文件管理、拉取和回滚下发

本文涉及的产品
日志服务 SLS,月写入数据量 50GB 1个月
应用型负载均衡 ALB,每月750个小时 15LCU
网络型负载均衡 NLB,每月750个小时 15LCU
简介:

前沿:

最近搞集群配置系统,正在搞配置文件的备份,中心点上传,文件hash记录,配置文件的下发回滚。 这里要用到saltstack cp模块,来实现对 master minion文件的传输及管理。

       这里说下,我主要的实现方式,文件备份是通过自写的模块备份,然后调用cp.push来拉取到master备份。下发回滚的话,用的额是get_file的方式,从master拖过去。   其实saltstack本身就含有文件的备份,但是为了多方面着想,多写了一个逻辑,存放到master点。

       这里提一句,有朋友很纳闷,jinja2渲染好的配置文件为啥不用sls的模式,而用get_url的模式。  个人觉得,如果是初始化配置,sls很是方便,但是对于经常有变动的服务,反而通过mongodb记录数据,然后web api接口渲染配置文件来的更方便。

(备份回滚思路,有兴趣的朋友可以参考下)  

wKioL1METx-AtS86AAF8KkgkgeQ255.jpg

说的有点乱,具体的自己看文档 !

get_file


cp.get_file用来从master下载文件到客户端,可以外加几个参数,比如没有文件夹,创建文件夹的makedirs=True ,压缩的gzip参数。  

语法如下:


1
salt  '*'  cp.get_file salt: / / rr  / etc / rr



get_url


cp.get_url可以从一个URL地址下载文件,URL可以是msater上的路径(salt://),也可以是http网址。


1
2
salt  '*'  cp.get_url salt: //my/file /tmp/mine
salt  '*'  cp.get_url http: //xiaorui.cc/good.txt  /tmp/good.txt


个人觉得 get_template 没啥用处,用他还不如sls的推送。


通过saltstack cp 实现配置文件的下发 !


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
[root@ruifengyun _modules ]$ cd /srv
[root@ruifengyun /srv ]$ ll
s总用量  4
drwxr-xr-x  4  root root  4096   2 月  13  17 : 01  salt
[root@ruifengyun /srv ]$ cd salt
[root@ruifengyun salt ]$ ll
总用量  12
drwxr-xr-x  2  root root  4096   2 月  13  16 : 59  _grains
drwxr-xr-x  2  root root  4096   2 月  19  10 : 20  _modules
-rw-r--r--  1  root root    61   2 月  13  16 : 59  top.sls
[root@ruifengyun salt ]$ mkdir backup
[root@ruifengyun salt ]$ cd backup
[root@ruifengyun backup ]$ ll
总用量  0
[root@ruifengyun backup ]$ touch  111
[root@ruifengyun backup ]$ echo  111  > 1
[root@ruifengyun backup ]$ cat  1
111
[root@ruifengyun backup ]$
[root@ruifengyun backup ]$
[root@ruifengyun backup ]$ salt  'lvs150.xiaorui.cc'  cp.get_file salt: //backup/1 /root/1
lvs150.xiaorui.cc:
     /root/ 1
[root@ruifengyun backup ]$                                                     
[root@ruifengyun backup ]$
[root@ruifengyun backup ]$ salt  'lvs150.xiaorui.cc'  cp.get_url http: //10.58.101.248/api/wgetconf\?masterip\=10.2.20.111\&typemode\=reallist /root/keeprealist
lvs150.xiaorui.cc:
     /root/keeprealist
[root@ruifengyun backup ]$
[root@ruifengyun backup ]$ salt  'lvs150.xiaorui.cc'  cmd.run  "head /root/keeprealist"                                                                 
lvs150.xiaorui.cc:
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        
             virtual_server  10.2 . 20.11  80  {
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
                                 delay_loop  6
                                 lb_algo    wlc
                                 lb_kind    DR
                                 persistence_timeout   0
[root@ruifengyun backup ]$
[root@ruifengyun backup ]$


wKioL1MERyGTCOJCAAWbs51xYVY398.jpg

原文:http://rfyiamcool.blog.51cto.com/1030776/1360468


从minion拉取文件!


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
[root@ruifengyun backup ]$
[root@ruifengyun backup ]$ cat  / var / cache / salt / master / minions / lvs150.xiaorui.cc / files / root / keeprealist
[root@ruifengyun backup ]$
[root@ruifengyun backup ]$ salt  'lvs150.xiaorui.cc'  cp.push  / etc / keepalived / conf.d / wan / reallist.conf
lvs150.xiaorui.cc:
     True
[root@ruifengyun backup ]$ cat  / var / cache / salt / master / minions / lvs150.xiaorui.cc / files / etc / keepalived / conf.d / wan / reallist.conf|more
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      
         virtual_server  10.2 . 20.11  80  {
                             delay_loop  6
                             lb_algo    wlc
                             lb_kind    DR
                             persistence_timeout   0
                             protocol TCP
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
                             real_server  10.2 . 20.12  80   {
                                                 weight  1
                                                 inhibit_on_failure
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      
                                                     TCP_CHECK {
                                                         connect_timeout  10
                                                         nb_get_retry  3
                                                         delay_before_retry  3
                                                         connect_port  80
                                                     }



wKiom1MER3rSZt2TAAM_-ToznS4719.jpg


有兴趣的朋友可以改cp.push的源码,也就那几行。

原文:http://rfyiamcool.blog.51cto.com/1030776/1360468


[root@ruifengyun backup ]$ vim /usr/lib/python2.7/dist-packages/salt/modules/cp.py


我这里为了获取的方便,用basename来获取他的文件名。



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
def push(path):
     '' '
     Push a file from the minion up to the master, the file will be saved to
     the salt master  in  the master's minion files cachedir
     (defaults to / var /cache/salt/master/minions/files)
     Since  this  feature allows a minion to push a file up to the master server
     it  is  disabled by  default  for  security purposes. To enable add the option:
     file_recv: True
     to the master configuration and restart the master
     CLI Example::
         salt  '*'  cp.push /etc/fstab
     '' '
     path=os.path.basename(path)
     if  '../'  in  path or not os.path.isabs(path):
         return  False
     if  not os.path.isfile(path):
         return  False
     auth = _auth()
     load = { 'cmd' '_file_recv' ,
             'id' : __opts__[ 'id' ],
             'path' : path.lstrip(os.sep)}
     sreq = salt.payload.SREQ(__opts__[ 'master_uri' ])
     with  salt.utils.fopen(path)  as  fp_:
         while  True:
             load[ 'loc' ] = fp_.tell()
             load[ 'data' ] = fp_.read(__opts__[ 'file_buffer_size' ])
             if  not load[ 'data' ]:
                 return  True
             ret = sreq.send( 'aes' , auth.crypticle.dumps(load))
             if  not ret:
                 return  ret




就这样了!   测试的模式用的是cli,大家在运维平台上可以使用api。






 本文转自 rfyiamcool 51CTO博客,原文链接:http://blog.51cto.com/rfyiamcool/1360468,如需转载请自行联系原作者




相关实践学习
SLB负载均衡实践
本场景通过使用阿里云负载均衡 SLB 以及对负载均衡 SLB 后端服务器 ECS 的权重进行修改,快速解决服务器响应速度慢的问题
负载均衡入门与产品使用指南
负载均衡(Server Load Balancer)是对多台云服务器进行流量分发的负载均衡服务,可以通过流量分发扩展应用系统对外的服务能力,通过消除单点故障提升应用系统的可用性。 本课程主要介绍负载均衡的相关技术以及阿里云负载均衡产品的使用方法。
相关文章
|
测试技术 Shell 应用服务中间件
|
消息中间件 监控 网络协议
SaltStack安装Apache/Mysql/PHP部署Wordpress
SaltStack是一个服务器基础架构集中化管理平台,具备配置管理、远程执行、监控等功能,基于Python语言实现,结合轻量级消息队列(ZeroMQ)与Python第三方模块(Pyzmq、PyCrypto、Pyjinjia2、python-msgpack和PyYAML等)构建。 SaltStack 采用 C/S模式,server端就是salt的master,client端就是minion,minion与master之间通过ZeroMQ消息队列通信。 master监听4505和4506端口,4505对应的是ZMQ的PUB system,用来发送消息,4506对应的是REP system是来接受
179 0