HA Cluster—Corosync+NFS实现LAMP高可用

本文涉及的产品
RDS MySQL Serverless 基础系列,0.5-2RCU 50GB
云数据库 RDS MySQL,集群系列 2核4GB
推荐场景:
搭建个人博客
云数据库 RDS MySQL,高可用系列 2核4GB
简介:

本次试验是基于Corosync和NFS对LAMP做高可用;

在一台宕机后另一台可以接替。

很多原理性的理论:HA Cluster—heartbeat v2基于crm配置有介绍;这里就不作介绍;直接进入配置阶段。

一、准备环境

服务器 IP 主机名
httpd+php+mysql 192.168.0.111 node1.soul.com
httpd+php+mysql 192.168.0.112 node2.soul.com
NFS 192.168.0.113 nfs.soul.com
VIP
192.168.0.222
  • 同步时间

1
2
3
4
5
6
7
8
#为了方便,这里单独使用了一台ansible机器,并非实验必须
[root@ansible ~] # ansible nodes -a "date"
node1.soul.com | success | rc = 0  >>
Wed Apr  23  09 : 36 : 53  CST  2014
node2.soul.com | success | rc = 0  >>
Wed Apr  23  09 : 36 : 53  CST  2014
nfs.soul.com | success | rc = 0  >>
Wed Apr  23  09 : 36 : 53  CST  2014
  • 对应的机器上安装软件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
[root@node1 ~] # rpm -q httpd php
httpd - 2.2 . 15 - 29.el6 .centos.x86_64
php - 5.3 . 3 - 26.el6 .x86_64
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
[root@node2 ~] # rpm -q httpd php
httpd - 2.2 . 15 - 29.el6 .centos.x86_64
php - 5.3 . 3 - 26.el6 .x86_64
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
#两台机器都需要
[root@node1 ~] # chkconfig httpd off
[root@node1 ~] # chkconfig --list httpd
httpd            0 :off    1 :off    2 :off    3 :off    4 :off    5 :off    6 :off
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#分别在两台机器上安装mysql
#注意的是如果在node1上操作,那node2上就不要初始化数据库了
#这里数据库的存储目录需要指定为NFS的共享目录;且初始化的时候需要先挂载NFS
[root@node1 ~] # mount
none on  / proc / sys / fs / binfmt_misc  type  binfmt_misc (rw)
sunrpc on  / var / lib / nfs / rpc_pipefs  type  rpc_pipefs (rw)
192.168 . 0.113 : / webstore on  / share  type  nfs (rw,vers = 4 ,addr = 192.168 . 0.113 ,clientaddr = 192.168 . 0.111 )
#如上所示,两台都是如此
  • 安装nfs

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[root@nfs ~] # service nfs start
Starting NFS services:                                     [  OK  ]
Starting NFS quotas:                                       [  OK  ]
Starting NFS mountd:                                       [  OK  ]
Starting NFS daemon:                                       [  OK  ]
Starting RPC idmapd:                                       [  OK  ]
[root@nfs ~] #
[root@nfs ~] # exportfs -v
[root@nfs ~] # exportfs -v
/ webstore        192.168 . 0.111 (rw,wdelay,no_root_squash,no_subtree_check)
/ webstore        192.168 . 0.112 (rw,wdelay,no_root_squash,no_subtree_check
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
#在httpd服务器上查看
[root@node1 ~] # showmount -e 192.168.0.113
Export  list  for  192.168 . 0.113 :
/ webstore  192.168 . 0.112 , 192.168 . 0.111
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      
#所有服务都准备测试好以后,全部关闭,并关闭开启自动启动。
#NFS服务需要开机启动;否则挂不上

二、安装配置corosync和pacemaker

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
#这里以node1操作
[root@node1 ~] # rpm -q corosync pacemaker
corosync - 1.4 . 1 - 17.el6 .x86_64
pacemaker - 1.1 . 10 - 14.el6 .x86_64
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      
#配置corosync
[root@node1 ~] # cp /etc/corosync/corosync.conf.example /etc/corosync/corosync.conf
[root@node1 ~] # vim /etc/corosync/corosync.conf
# Please read the corosync.conf.5 manual page
compatibility: whitetank
totem {
         version:  2             #版本号
         secauth: on            #认证
         threads:  0             #认证时并行线程数
         interface {
                 ringnumber:  0     #环号码
                 bindnetaddr:  192.168 . 0.0     #绑定的网络
                 mcastaddr:  226.94 . 40.1       #多播地址
                 mcastport:  5405              #多播端口
                 ttl:  1                       #发送次数
         }
}
logging {
         fileline: off
         to_stderr: no                 #输入到标准错误
         to_logfile: yes               #启用日志
         to_syslog: no                 #发送到系统日志
         logfile:  / var / log / cluster / corosync.log  #日志路径
         debug: off                    #是否开启debug
         timestamp: on                 #是否开启时间戳
         logger_subsys {
                 subsys: AMF
                 debug: off
         }
}
amf {
         mode: disabled
}
service {
         ver:     0                 #版本号
         name:   pacemaker         #开启自动启动pacemaker
}
aisexec {
         user:   root             #运行时用户
         group:  root
}
- -  INSERT  - -
  • 生成认证密钥

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
[root@node1 corosync] # corosync-keygen
Corosync Cluster Engine Authentication key generator.
Gathering  1024  bits  for  key  from  / dev / random.
Press keys on your keyboard to generate entropy.
Writing corosync key to  / etc / corosync / authkey.
#可能会有与随机数不够导致需要敲键盘;可以选择敲键盘;也可以使用伪随机数
#建议没事多敲敲键盘;即保证了安全;有锻炼了身体
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
#拷贝authkey corosync.conf到node2
[root@node1 corosync] # ls
authkey        corosync.conf.example       service.d
corosync.conf  corosync.conf.example.udpu  uidgid.d
#
[root@node1 corosync] # scp -p authkey corosync.conf node2:/etc/corosync/
authkey                                                      100 %   128      0.1KB / s    00 : 00
corosync.conf                                                100 %   520      0.5KB / s    00 : 00
[root@node1 corosync] #
#注意权限
  • 启动测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
[root@node1 ~] # service corosync start
Starting Corosync Cluster Engine (corosync):               [  OK  ]
[root@node1 ~] # ssh node2 'service corosync start'
Starting Corosync Cluster Engine (corosync): [  OK  ]
[root@node1 ~] #
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
#验证是否启动成功
#验证启动是否正常
[root@node1 ~] # grep -e "Corosync Cluster Engine" -e "configuration file" /var/log/cluster/corosync.log
Apr  23  11 : 48 : 40  corosync [MAIN  ] Corosync Cluster Engine ( '1.4.1' ): started  and  ready to provide service.
Apr  23  11 : 48 : 40  corosync [MAIN  ] Successfully read main configuration  file  '/etc/corosync/corosync.conf' .
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
#验证初始化成员节点通知是否正常发出
[root@node1 ~] # grep TOTEM /var/log/cluster/corosync.log
Apr  23  11 : 48 : 40  corosync [TOTEM ] Initializing transport (UDP / IP Multicast).
Apr  23  11 : 48 : 40  corosync [TOTEM ] Initializing transmit / receive security: libtomcrypt SOBER128 / SHA1HMAC (mode  0 ).
Apr  23  11 : 48 : 40  corosync [TOTEM ] The network interface [ 192.168 . 0.111 is  now up.
Apr  23  11 : 48 : 41  corosync [TOTEM ] Process pause detected  for  879  ms, flushing membership messages.
Apr  23  11 : 48 : 41  corosync [TOTEM ] A processor joined  or  left the membership  and  a new membership was formed.
Apr  23  11 : 48 : 53  corosync [TOTEM ] A processor joined  or  left the membership  and  a new membership was formed.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
#查看pacemaker是否正常启动
[root@node1 ~] # grep pcmk_startup /var/log/cluster/corosync.log
Apr  23  11 : 48 : 40  corosync [pcmk  ] info: pcmk_startup: CRM: Initialized
Apr  23  11 : 48 : 40  corosync [pcmk  ] Logging: Initialized pcmk_startup
Apr  23  11 : 48 : 40  corosync [pcmk  ] info: pcmk_startup: Maximum core  file  size  is 18446744073709551615
Apr  23  11 : 48 : 40  corosync [pcmk  ] info: pcmk_startup: Service:  9
Apr  23  11 : 48 : 40  corosync [pcmk  ] info: pcmk_startup: Local hostname: node1.soul.com
[root@node1 ~] #
  • 安装crmsh和pssh两个包

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
[root@node1 ~] # scp -p pssh-2.3.1-2.el6.x86_64.rpm crmsh-1.2.6-4.el6.x86_64.rpm node2:/root
pssh - 2.3 . 1 - 2.el6 .x86_64.rpm                                  100 %    49KB   48.8KB / s    00 : 00
crmsh - 1.2 . 6 - 4.el6 .x86_64.rpm                                 100 %   484KB  483.7KB / s    00 : 00
[root@node1 ~] #
                                                                                                                                                                                                                                                                                                                                                                                                                                                          
[root@node1 ~] # yum -y install crmsh-1.2.6-4.el6.x86_64.rpm pssh-2.3.1-2.el6.x86_64.rpm
                                                                                                                                                                                                                                                                                                                                                                                                                                                          
[root@node2 ~] # yum -y install crmsh-1.2.6-4.el6.x86_64.rpm pssh-2.3.1-2.el6.x86_64.rpm
                                                                                                                                                                                                                                                                                                                                                                                                                                                        
#安装完成后即可使用crm命令来查看
[root@node1 ~] # crm status
Last updated: Wed Apr  23  11 : 57 : 29  2014
Last change: Wed Apr  23  11 : 49 : 04  2014  via crmd on node2.soul.com
Stack: classic openais (with plugin)
Current DC: node2.soul.com  -  partition with quorum
Version:  1.1 . 10 - 14.el6 - 368c726
2  Nodes configured,  2  expected votes
0  Resources configured
Online: [ node1.soul.com node2.soul.com ]
                                                                                                                                                                                                                                                                                                                                                                                                                                                      
#crm的用法:
[root@node1 ~] # crm
crm(live) # help
#就是这样,一句话也说不清楚,需要慢慢研究,
This  is  crm shell, a Pacemaker command line interface.
Available commands:
     cib              manage shadow CIBs
     resource         resources management
     configure        CRM cluster configuration
     node             nodes management
     options          user preferences
     history          CRM cluster history
     site             Geo - cluster support
     ra               resource agents information center
     status           show cluster status
     help ,?           show  help  ( help  topics  for  list  of topics)
     end,cd,up        go back one level
     quit,bye,exit    exit the program

一切准备就绪后;下面就是配置资源了,最繁琐的也是这里

三、配置高可用集群资源

首先做个简单的规划;需要配置哪些资源,先后次序后续启动时很重要的:

1、配置VIP

2、配置NFS共享存储

3、配置httpd服务

4、配置mysql服务

5、配置一个资源组;把上述资源加入该组

所需要的命令或需要使用的代理大致:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16