开发者社区> 余二五> 正文
阿里云
为了无法计算的价值
打开APP
阿里云APP内打开

DRDB实现mariadb高可用

2017-11-15 2441

简介：
+关注继续查看

------本文大纲

简介

DRBD资源属性

原理

安装DRDB

DRDB实现高可用

============

一、简介

DRBD是一种块设备，可以被用于高可用(HA)之中。它类似于一个网络RAID-1功能。当你将数据写入本地文件系统时，数据还将会被发送到网络中另一台主机上。以相同的形式记录在一个文件系统中。本地(主节点)与远程主机(备节点)的数据可以保证实时同步。当本地系统出现故障时，远程主机上还会保留有一份相同的数据，可以继续使用。在高可用(HA)中使用DRBD功能，可以代替使用一个共享盘阵。因为数据同时存在于本地主机和远程主机上。切换时，远程主机只要使用它上面的那份备份数据，就可以继续进行服务了。

二、DRBD资源属性

resource name

可以使用除空白字符外的任意ACSII表中的字符；

drbd设备

drbd的设备的访问路径，设备文件/dev/drbd#；

disk

各节点为组成此drbd设备所提供的块设备，通常是一个磁盘分区；

网络属性

节点间为了实现跨主机磁盘镜像而使用的网络配置；

注意：用户空间工具与drdb与内核中使用的模块版本要保持一致，只有在使用drbdadm工具时才会读取配置文件，对多个资源的公共配置，可以提取出来只配置一次，通常保存在Common中，此外还有global配置，这种配置跟资源本身没有关系的；

三、原理

wKiom1NhbnjyRON3AAF1bsI5qmM739.jpg

需要知道，DRBD是工作在系统内核空间，而不是用户空间，它直接复制的是二进制数据，这是它速度快的根本原因。

其次，DRBD至少需要两个节点来工作，一主一次。

DRBD的文件同步过程和普通复制过程的不同：

DRBD在数据进入Buffer Cache时，先经过DRBD这一层，复制一份数据经过TCP/IP协议封装，发送到另一个节点上，另一个节点通过TCP/IP协议来接受复制过来的数据，同步到次节点的DRBD设备上。

  • DRBD底层设备支持

DRBD需要构建在底层设备之上，然后构建出一个块设备出来。对于用户来说，一个DRBD设备，就像是一块物理的磁盘，可以在上面内创建文件系统。DRBD所支持的底层设备有以下这些类：

1、一个磁盘，或者是磁盘的某一个分区；

2、一个soft raid 设备；

3、一个LVM的逻辑卷；

  • DRBD在远程传输上支持三种模式：

协议A:异步复制协议

只要主节点完成本地写操作就认为写操作完成，并且需要复制的数据包会被存放本地TCP发送缓存中，当发生failover故障，数据可能会丢失。

协议B：内存同步（半同步，semi-synchronous）复制协议

当本地磁盘的写己经完成，并且复制数据包己经到达对应从节点，此时主节点才认为磁盘写入己经完成，failover不会导致数据丢失，但当所有节点同时出现故障时，主节点上多数最新写入的数据可能丢失。

协议C:同步复制

只有在本地和远程磁盘都确定写入己完成后，主节点才会认为写入完成。这样可确保发生单点故障时不会导致任何数据丢失，如果发生数据丢失，那也只会在所有节点同时存在错误存储时才会发生这种情况

  • 角色、模式

    • 角色
      在drbd构造的集群中，资源具有角色的概念，分别为primary和secondary。
      所有设为primary的资源将不受限制进行读写操作。可以创建文件系统，可以使用裸设备,甚至直接io。所有设为secondary的设备中不能挂载，不能读写

    • 模式

      drbd也有drbd mode：单主模型（主从），双主模型
      在单主模型下drbd可以使用任意的文件系统
      单在双主模型下只能使用集群文件系统，常用的开源的集群文件系统有:ocfs2和gfs2

四、安装DRDB

前提：

设置主机名（要与uname -n一致，主机名可解析）

时间同步

无密钥登录

本机环境

系统 主机名 ip地址
Centos6.5 x86_64 essun.node1.com 192.168.1.109
Centos6.5 x86_64 essun.node2.com 192.168.1.108

注：

内核模块程序包一定要跟你的系统内核保持一致，uname -r查看你的内核版本，内核模块的版本必须要严格对应，而用户空间的模块就不那么严格要求了，两个节点的时间也需要保持一致性；

第一步、下载与内核版本相应的内核模块与用户空间工具（地址：http://packages.atrpms.net/dist/el6/drbd/）

1
2
3
4
5
[root@essun download]# uname -r
2.6.32-431.el6.x86_64
[root@essun download]# ls
drbd-8.4.3-33.el6.x86_64.rpm  drbd-kmdl-2.6.32-431.el6-8.4.3-33.el6.x86_64.rpm
[root@essun download]# yum isntall drbd-kmdl-2.6.32-431.el6-8.4.3-33.el6.x86_64.rpm drbd-8.4.3-33.el6.x86_64.rpm -y

  • 修改配置文件（/etc/drbd.d/global_common.conf）

  • 1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    [root@essun drbd.d]# vim global_common.conf
    [root@essun drbd.d]# cat global_common.conf
    global {
        usage-count no;
    #  是否参加DRBD使用者统计.默认是yes
        # minor-count dialog-refresh disable-ip-verification
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
    common {
        handlers {
            # These are EXAMPLE handlers only.
            # They may have severe implications,
            # like hard resetting the node under certain circumstances.
            # Be careful when chosing your poison.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
            pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
            pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
            local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";
            # fence-peer "/usr/lib/drbd/crm-fence-peer.sh";
            # split-brain "/usr/lib/drbd/notify-split-brain.sh root";
            # out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root";
            # before-resync-target "/usr/lib/drbd/snapshot-resync-target-lvm.sh -p 15 -- -c 16k";
            # after-resync-target /usr/lib/drbd/unsnapshot-resync-target-lvm.sh;
        }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
        startup {
            # wfc-timeout degr-wfc-timeout outdated-wfc-timeout wait-after-sb
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
            #wfc-timeout 120;
                     #(等待另一个节点上线的超时时长)
           # degr-wfc-timeout 240;
                    #(等待超时后做降级处理)
            #outdated-wfc-timeout
                    #(过期的等待超时)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
                    # wait-after-sb
                    #(脑裂之后等待多长时长)
        }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
        options {
            # cpu-mask on-no-data-accessible
        }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
        disk {
            # size max-bio-bvecs on-io-error fencing disk-barrier disk-flushes
            on-io-error detach;
                    # 同步错误的做法是分离
            # disk-drain md-flushes resync-rate resync-after al-extents
                    # c-plan-ahead c-delay-target c-fill-target c-max-rate
                    # c-min-rate disk-timeout
        }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
        net {
            protocol C;
     # 使用协议C.表示收到远程主机的写入确认后,则认为写入完成.
            cram-hmac-alg "sha1";
            shared-secret "drdb.essun.drdb.org";
     # 设置主备机之间通信使用的信息算法.
            # protocol timeout max-epoch-size max-buffers unplug-watermark
            # connect-int ping-int sndbuf-size rcvbuf-size ko-count
            # allow-two-primaries cram-hmac-alg shared-secret after-sb-0pri
            # after-sb-1pri after-sb-2pri always-asbp rr-conflict
            # ping-timeout data-integrity-alg tcp-cork on-congestion
            # congestion-fill congestion-extents csums-alg verify-alg
            # use-rle
        }
        syncer {
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
            rate 1000M;
    }
     # 设置主备节点同步时的网络速率最大值,单位是字节.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  
    }

    提供物理块设备（两个节点的分区号最好相同），不需要格式化

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
[root@essun drbd.d]# fdisk -cu /dev/s
scd0      sda1      sg0       shm/      snd/      stderr    stdout
sda       sda2      sg1       snapshot  sr0       stdin     systty
[root@essun drbd.d]# fdisk -cu /dev/sda
Command (m for help): p
Disk /dev/sda: 128.8 GB, 128849018880 bytes
255 heads, 63 sectors/track, 15665 cylinders, total 251658240 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006c47d
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048      411647      204800   83  Linux
/dev/sda2          411648   126240767    62914560   8e  Linux LVM
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 3
First sector (126240768-251658239, default 126240768):
Using default value 126240768
Last sector, +sectors or +size{K,M,G} (126240768-251658239, default 251658239): +10G
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[root@essun drbd.d]# partx -a /dev/sda
BLKPG: Device or resource busy
error adding partition 1
BLKPG: Device or resource busy
error adding partition 2
[root@essun drbd.d]# partx -a /dev/sda
BLKPG: Device or resource busy
error adding partition 1
BLKPG: Device or resource busy
error adding partition 2
BLKPG: Device or resource busy
error adding partition 3
[root@essun drbd.d]# partx -a /dev/sda
BLKPG: Device or resource busy
error adding partition 1
BLKPG: Device or resource busy
error adding partition 2
BLKPG: Device or resource busy
error adding partition 3
[root@essun drbd.d]# partx -a /dev/sda
BLKPG: Device or resource busy
error adding partition 1
BLKPG: Device or resource busy
error adding partition 2
BLKPG: Device or resource busy
error adding partition 3

  • 定义DRBD磁盘资源

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[root@essun drbd.d]# vim dbresouce.res
resource dbresource {
        on essun.node1.com { #on说明在哪个节点上，跟uname -n保持一致
                device /dev/drbd0; #在磁盘上表现的drbd叫什么名；
                disk   /dev/sda3; #所使用的物理磁盘设备是哪个；
                address 192.168.1.109:7789; #在node1这个节点上监听的套接字,默认监听在7789端口上；
                meta-disk internal; #保存drbd源数据信息，表示就放在自己的磁盘区分上，也可以放在外部的磁盘上；
        }
        on essun.node2.com{
                device /dev/drbd0;
                disk /dev/sda3;
                address 192.168.1.108:7789;
                meta-disk internal;
        } 
}

  • 将这两个文件拷贝到node1中的相同目录下

1
2
3
4
5
[root@essun drbd.d]# scp dbresouce.res global_common.conf essun.node1.com:/etc
/drbd.d/
dbresouce.res                               100%  259     0.3KB/s   00:00
global_common.conf                          100% 2044     2.0KB/s   00:00
[root@essun drbd.d]#

  • 在各自的节点上初始化资源

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
[root@essun drbd.d]# drbdadm create-md dbresource
#初始化资源列表
Writing meta data...
initializing activity log
NOT initializing bitmap
lk_bdev_save(/var/lib/drbd/drbd-minor-0.lkbd) failed: No such file or directory
New drbd meta data block successfully created.
lk_bdev_save(/var/lib/drbd/drbd-minor-0.lkbd) failed: No such file or directory
[root@essun drbd.d]# service drbd start
#启动服务
Starting DRBD resources: [
     create res: dbresource
   prepare disk: dbresource
    adjust disk: dbresource
     adjust net: dbresource
]
.
[root@essun drbd.d]# drbd-overview
#查看drbd的状态
  0:dbresource/0  Connected Secondary/Secondary Inconsistent/Inconsistent C r-----
[root@essun drbd.d]# watch -n 1 'cat /proc/drbd'
#实时查看drbd的状态
Every 1.0s: cat /proc/drbd                                                     Thu May  1 08:48:23 2014
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by gardner@, 2013-11-29 12:28:00
 0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r-----
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:10485404

上面表示两个节点的drbd都是从的，这是因为我们并没有对其中任何一个提升主的。

  • 将其中一个节点的DRBD资源提升为主资源

1
2
3
4
5
6
[root@essun drbd.d]# drbdadm primary --force dbresource
#将当前节点的drbd资源提升为主资源
[root@essun drbd.d]# drbd
[root@essun drbd.d]# drbd-overview
  0:dbresource/0  SyncSource Primary/Secondary UpToDate/Inconsistent C r---n-
    [>...................] sync'ed:  6.1% (9624/10236)M

  • 初始化之后（角色己经发生改变了）

1
2
[root@essun drbd.d]# drbd-overview
  0:dbresource/0  Connected Primary/Secondary UpToDate/UpToDate C r-----

  • 格式化DRBD

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
[root@essun drbd.d]# mke2fs -t ext4 /dev/drbd0
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
655360 inodes, 2621351 blocks
131067 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
Writing inode tables: done                      
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 28 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
[root@essun drbd.d]# mount /dev/drbd0 /mnt
[root@essun drbd.d]# cd mnt
-bash: cd: mnt: No such file or directory
[root@essun drbd.d]# cd /mnt
[root@essun mnt]# ls
lost+found
[root@essun mnt]# cp /etc/inittab .
[root@essun mnt]# ls
inittab  lost+found
[root@essun mnt]#

  • 提升另一个节点的DRBD资源

1
2
3
4
5
6
7
8
9
10
11
12
[root@essun /]# umount /mnt
#卸载drbd
[root@essun /]# drbdadm secondary dbresource
# 降级DRBD资源
[root@essun /]# drbd-overview
  0:dbresource/0  Connected Secondary/Secondary UpToDate/UpToDate C r-----
#以下是在另一个节点操作
[root@essun ~]# drbdadm primary dbresource
[root@essun ~]# mount /dev/drbd0 /mnt
[root@essun ~]# cd /mnt
[root@essun mnt]# ls
inittab  lost+found

如果想让DRBD能够自动切换的话，就要借助于messages layer程序了。

首先在对DRBD降级，停止服务，禁止开机自动启动

1
2
3
4
5
6
7
8
9
[root@essun /]# umount /mnt
[root@essun /]# drbdadm secondary dbresource
[root@essun /]# drbd-overview
  0:dbresource/0  Connected Secondary/Secondary UpToDate/UpToDate C r-----
[root@essun /]# service drbd stop
Stopping all DRBD resources: .
[root@essun /]# chkconfig drbd off
[root@essun /]# chkconfig --list drbd
drbd            0:off   1:off   2:off   3:off   4:off   5:off   6:off

第二步、安装mariadb

1
2
3
4
5
6
7
[root@essun download]# tar xf mariadb-10.0.10-linux-x86_64.tar.gz
[root@essun download]# mv mariadb-10.0.10-linux-x86_64 /usr/local
[root@essun download]# cd /usr/local/
[root@essun local]# ln -s mariadb-10.0.10-linux-x86_64 mysql
[root@essun local]# groupadd -r mysql
[root@essun local]# useradd -r -g mysql mysql
[root@essun local]# chown -R mysql.mysql mari/

  • 启动DRBD服务，提升主资源

1
2
3
4
5
6
7
8
9
10
11
12
[root@essun ~]# service drbd start
Starting DRBD resources: [
     create res: dbresource
   prepare disk: dbresource
    adjust disk: dbresource
     adjust net: dbresource
]
.
[root@essun local]# drbd-overview
  0:dbresource/0  Connected Secondary/Secondary UpToDate/UpToDate C r-----
[root@essun local]# drbd-overview
  0:dbresource/0  Connected Primary/Secondary UpToDate/UpToDate C r-----

  • 挂载数据目录

1
2
3
4
5
6
7
8
9
10
11
12
[root@essun local]# mkdir /mariadb
[root@essun local]# mount /dev/drbd0 /mariadb/
[root@essun local]# cd /mariadb/
[root@essun mariadb]# mkdir {data,logs}
[root@essun mariadb]# ls
data   logs  lost+found
[root@essun /]# chown -R mysql.mysql /mariadb/
[root@essun /]# ll /mariadb/
total 24
drwxr-xr-x 2 mysql mysql  4096 May  1 11:09 data
drwxr-xr-x 2 mysql mysql  4096 May  1 11:09 logs
drwx------ 2 mysql mysql 16384 May  1 09:42 lost+found

注：

此目录在另一个节点上也要一样。

  • 修改配置文件与添加启动脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
[root@essun mari]# cp support-files/my-large.cnf /etc/cnf
[root@essun mari]# vim /etc/my.cnf
[root@essun mari]# grep -v "^#" /etc/my.cnf |grep -v "
[client]
port        = 3306
socket      = /tmp/mysql.sock
[mysqld]
port        = 3306
socket      = /tmp/mysql.sock
skip-external-locking
key_buffer_size = 256M
max_allowed_packet = 1M
table_open_cache = 256
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size= 16M
thread_concurrency = 8
datadir=/mariadb/data
log-bin=/mariadb/logs/mysql-bin
binlog_format=mixed
server-id   = 1
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[myisamchk]
key_buffer_size = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
[root@essun mari]# cp support-files/mysql.server /etc/rc.d/init.d/mari
[root@essun mari]# chmod +x /etc/rc.d/init.d/mari
[root@essun mari]# chkconfig --add mari
[root@essun mari]# chkconfig mari off

  • 初始化数据库,并登录测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
[root@essun mari]# scripts/mysql_install_db --user=mysql --datadir=/mariadb/data/
Installing MariaDB/MySQL system tables in '/mariadb/data/' ...
140501 11:22:15 [Note] InnoDB: The InnoDB memory heap is disabled
140501 11:22:15 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
140501 11:22:15 [Note] InnoDB: Compressed tables use zlib 1.2.3
........省略中........
Please report any problems at http://mariadb.org/jira
The latest information about MariaDB is available at http://mariadb.org/.
You can find additional information about the MySQL part at:
http://dev.mysql.com
Support MariaDB development by buying support/new features from
SkySQL Ab. You can contact us about this at sales@skysql.com.
Alternatively consider joining our community based development effort:
http://mariadb.com/kb/en/contributing-to-the-mariadb-project/
[root@essun local]# service mari start
Starting MySQL........                                     [  OK  ]
[root@essun local]# /usr/local/mysql/bin/mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.0.10-MariaDB-log MariaDB Server
Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+
4 rows in set (0.00 sec)
MariaDB [(none)]>

  • 到另一个节点测试一下，先停止服务，卸载、降级

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
[root@essun local]# service mari stop
Shutting down MySQL..                                      [  OK  ]
[root@essun local]# umount /mariadb/
[root@essun local]# drbdadm secondary dbresource
[root@essun local]# drbd-overview
  0:dbresource/0  Connected Secondary/Secondary UpToDate/UpToDate C r-----
[root@essun local]# scp /etc/my.cnf essun.node1.com:/etc/
my.cnf                                                               100% 4936     4.8KB/s   00:00
[root@essun local]# scp -p /etc/rc.d/init.d/mari essun.node1.com:/etc/rc.d/init.d/
mari                                                                 100%   11KB  11.4KB/s   00:00
[root@essun local]#
==============以下是另一个节点的操作
[root@essun ~]# mkdir /mariadb
[root@essun ~]# drbdadm primary dbresource
[root@essun ~]# drbd-overview
  0:dbresource/0  Connected Primary/Secondary UpToDate/UpToDate C r-----
[root@essun ~]# chown -R mysql.mysql /mariadb/
[root@essun ~]# mount /dev/drbd0 /mariadb/
[root@essun ~]# cd /mariadb/
[root@essun mariadb]# ll
total 24
drwxr-xr-x 5 mysq mysql  4096 May  1 11:29 data
drwxr-xr-x 2 mysq mysql  4096 May  1 11:29 logs
drwx------ 2 mysq mysql 16384 May  1 09:42 lost+found
[root@essun /]# service mari start
Starting MySQL.......                                      [  OK  ]
[root@essun /]# /usr/local/mysql/bin/mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.0.10-MariaDB-log MariaDB Server
Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+
4 rows in set (0.03 sec)
MariaDB [(none)]>

  • 两个节点都要停止服务

1
2
3
4
5
6
7
8
9
10
11
[root@essun /]# service mari stop
Shutting down MySQL..                                      [  OK  ]
[root@essun /]# chkconfig mari off
[root@essun /]# chkconfig --list mari
mari            0:off   1:off   2:off   3:off   4:off   5:off   6:off
[root@essun /]# service drbd stop
Stopping all DRBD resources:
.
[root@essun /]# chkconfig drbd off
[root@essun /]# chkconfig --list drbd
drbd            0:off   1:off   2:off   3:off   4:off   5:off   6:off

第三步、安装corosync、pacemaker

1
2
[root@essun download]# yum install corosync pacemaker -y
[root@essun download]# yum install crmsh-1.2.6-4.el6.x86_64.rpm pssh-2.3.1-2.el6.x86_64.rpm -y

  • 修改配置文件，生成密钥

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
[root@essun /]# cd /etc/corosync/
[root@essun corosync]# ls
corosync.conf.example       service.d
corosync.conf.example.udpu  uidgid.d
[root@essun corosync]# cp corosync.conf.example corosync.conf
[root@essun corosync]# vim corosync.conf
[root@essun corosync]# cat corosync.conf
# Please read the corosync.conf.5 manual page
compatibility: whitetank
totem {
    version: 2
    secauth: off
    threads: 0
    interface {
        ringnumber: 0
        bindnetaddr: 192.168.1.1
        mcastaddr: 226.94.1.1
        mcastport: 5405
        ttl: 1
    }
}
logging {
    fileline: off
    to_stderr: no
    to_logfile: yes
    to_syslog: no
    logfile: /var/log/cluster/corosync.log
    debug: off
    timestamp: on
    logger_subsys {
        subsys: AMF
        debug: off
    }
}
amf {
    mode: disabled
}
service {
        name:   pacemaker
        ver:    0
}
aisexce {
        user:   root
        group:  root
}
[root@essun corosync]# mv /dev/random /dev/x
[root@essun corosync]# mv /dev/urandom /dev/random
[root@essun corosync]# corosync-keygen
Corosync Cluster Engine Authentication key generator.
Gathering 1024 bits for key from /dev/random.
Press keys on your keyboard to generate entropy.
Writing corosync key to /etc/corosync/authkey.
[root@essun corosync]# mv /dev/random /dev/urandom
[root@essun corosync]# mv /dev/x /dev/random

  • 将这两个文件复制到另一个节点上一份

1
2
3
4
[root@essun corosync]# scp authkey corosync.conf essun.node1.com:/etc/corosync/
authkey                100%  128     0.1KB/s   00:00
corosync.conf          100%  554     0.5KB/s   00:00
[root@essun corosync]#

  • 启动服务（两个节点都要启动服务）

1
2
3
4
5
6
7
8
9
10
11
12
[root@essun corosync]# service corosync start
Starting Corosync Cluster Engine (corosync):            [  OK  ]
[root@essun corosync]# crm status
Last updated: Thu May  1 10:05:25 2014
Last change: Thu May  1 10:05:21 2014 via crmd on essun.node2.com
Stack: classic openais (with plugin)
Current DC: essun.node2.com - partition with quorum
Version: 1.1.10-14.el6_5.3-368c726
2 Nodes configured, 2 expected votes
0 Resources configured
Online: [ essun.node1.com essun.node2.com ]
[root@essun corosync]#

  • 禁用stonith裁决，忽略投票规则

1
2
3
4
5
6
7
8
9
10
11
12
13
crm(live)configure# property stonith-enabled=false
crm(live)configure# property no-quorum-policy=ignore
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
node essun.node1.com
node essun.node2.com
property $id="cib-bootstrap-options" \
    dc-version="1.1.10-14.el6_5.3-368c726" \
    cluster-infrastructure="classic openais (with plugin)" \
    expected-quorum-votes="2" \
    stonith-enabled="false" \
    no-quorum-policy="ignore"

  • 定义VIP，mariadb服务

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
crm(live)configure# primitive vip ocf:heartbeat:IPaddr params ip=192.168.1.150 op monitor interval=20s timeout=20s on-fail=restart
crm(live)configure# verify
crm(live)configure# primitive myserver lsb:mari op monitor interval=20s timeout=20s on-fail=restart
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
crm(live)configure# show
node essun.node1.com \
    attributes standby="off"
node essun.node2.com
primitive myserver lsb:mari \
    op monitor interval="20s" timeout="20s" on-fail="restart"
primitive vip ocf:heartbeat:IPaddr \
    params ip="192.168.1.150" \
    op monitor interval="20s" timeout="20s" on-fail="restart"
property $id="cib-bootstrap-options" \
    dc-version="1.1.10-14.el6_5.3-368c726" \
    cluster-infrastructure="classic openais (with plugin)" \
    expected-quorum-votes="2" \
    stonith-enabled="false" \
    no-quorum-policy="ignore"

  • 配置drdb集群资源

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
crm(live)configure# primitive dbdrbd ocf:linbit:drbd params drbd_resource=dbresource op monitor role=Master interval=50s timeout=30s op monitor role=Slave interval=60s timeout=30s op start timeout=240s interval=0 op stop timeout=100s interval=0
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show
node essun.node1.com \
        attributes standby="off"
node essun.node2.com
node essun.node1.com \
        attributes standby="off"
node essun.node2.com
primitive dbdrbd ocf:linbit:drbd \
        params drbd_resource="dbresource" \
        op monitor role="Master" interval="50s" timeout="30s" \
        op monitor role="Slave" interval="60s" timeout="30s" \
        op start timeout="240s" interval="0" \
        op stop timeout="100s" interval="0"
primitive myserver lsb:mari \
        op monitor interval="20s" timeout="20s" on-fail="restart"
primitive vip ocf:heartbeat:IPaddr \
        params ip="192.168.1.150" \
        op monitor interval="20s" timeout="20s" on-fail="restart"
property $id="cib-bootstrap-options" \
        dc-version="1.1.10-14.el6_5.3-368c726" \
        cluster-infrastructure="classic openais (with plugin)" \
        expected-quorum-votes="2" \
        stonith-enabled="false" \
        no-quorum-policy="ignore"

drbd需要同时运行在两个节点上，但只能有一个节点（primary/secondary模型）是Master，而另一个节点为Slave；因此，它是一种比较特殊的集群资源，其资源类型为多态（Multi-state）clone类型，即主机节点有Master和Slave之分，且要求服务刚启动时两个节点都处于slave状态，一般来说，只要是定义克隆资源，首先其本身必须是主资源

1
2
3
4
5
6
crm(live)configure# ms ms_dbdrdb dbdrbd meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# show ms_dbdrdb
ms ms_dbdrdb dbdrbd \
    meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"

  • 到node1验证一下，看看是不是变成了主资源

1
2
3
4
5
6
7
8
9
10
11
12
[root@essun /]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:bf:c2:82 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.109/24 brd 255.255.255.255 scope global eth0
    inet 192.168.1.150/24 brd 255.255.255.255 scope global secondary eth0
    inet6 fe80::20c:29ff:febf:c282/64 scope link tentative dadfailed
       valid_lft forever preferred_lft forever

  • 定义文件系统资源（以便挂载DRBD的主资源，此处挂载的是mariadb的数据目录)

1
2
3
4
5
6
7
8
9
10
11
crm(live)configure# primitive dbstore ocf:heartbeat:Filesystem params device=/dev/drbd0 directory=/mariadb fstype=ext4 op monitor interval40s timeout=40s op start timeout=60s interval=0 op stop timeout=60 interval=0
crm(live)configure# verify
crm(live)configure# commit
WARNING: CIB changed in the meantime: won't touch it!
Do you still want to commit? y
crm(live)configure# show dbstore
primitive dbstore ocf:heartbeat:Filesystem \
    params device="/dev/drbd0" directory="/mariadb" fstype="ext4" \
    op monitor timeout="40s" interval="0" interval40s \
    op start timeout="60s" interval="0" \
    op stop timeout="60" interval="0

  • 结合drbd的特性，只能在drbd服务将某节点设置为Primary以后方可启动。因此，还需要为这两个资源建立排列约束和顺序约束。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
colocation vip_with_myserver inf: vip myserver
crm(live)configure# colocation myserver_with_dbststore inf: myserver
dbstore
crm(live)configure# colocation dbstore_with_ms_dbdrbd inf: dbstore ms_dbdrdb:Master
crm(live)configure# verify
crm(live)configure# commit
crm(live)configure# order ms_dbdrbd_before_dbstore inf: ms_dbdrdb:promote dbstore:start
crm(live)configure# verify
crm(live)configure# order vip_before_myserver inf: vip myserver
crm(live)configure# verify
crm(live)configure# order dbstore_before_myserver inf: dbstore:start myserver:start
crm(live)configure# verify
crm(live)configure# commit
crm(live)# status
Last updated: Thu May  1 15:29:04 2014
Last change: Thu May  1 15:28:55 2014 via crmd on essun.node2.com
Stack: classic openais (with plugin)
Current DC: essun.node2.com - partition with quorum
Version: 1.1.10-14.el6_5.3-368c726
2 Nodes configured, 2 expected votes
5 Resources configured
Online: [ essun.node1.com essun.node2.com ]
 vip    (ocf::heartbeat:IPaddr):        Started essun.node1.com
 myserver       (lsb:mari):     Started essun.node1.com
 Master/Slave Set: ms_dbdrdb [dbdrbd]
     Masters: [ essun.node1.com ]
     Slaves: [ essun.node2.com ]
 dbstore        (ocf::heartbeat:Filesystem):    Started essun.node1.com

  • 到节点node1上看一下maridb下是否己经挂载DRBD的主资源

1
2
3
4
[root@essun mariadb]# hostname
essun.node1.com
[root@essun mariadb]# ls
data  logs  lost+found

这己表示在DRBD己经挂载到node1的mariadb目下

第四步、测试高可用mariadb

  • 授权测试用户

1
2
3
4
5
6
7
8
[root@essun /]# /usr/local/mysql/bin/mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 10.0.10-MariaDB-log MariaDB Server
Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> grant all on *.* to root@'192.168.1.%' identified by 'mari';
Query OK, 0 rows affected (0.00 sec)

  • 使用vip登录测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
[root@essun mariadb]# hostname
essun.node2.com
[root@essun mariadb]# /usr/local/mysql/bin/mysql -uroot -p -h192.168.1.150
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 6
Server version: 10.0.10-MariaDB-log MariaDB Server
Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+
4 rows in set (0.08 sec)

  • 模拟节点node1故障

  • 1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    crm(live)node# standby essun.node1.com
    [root@essun ~]# crm status
    Last updated: Thu May  1 16:28:59 2014
    Last change: Thu May  1 16:27:34 2014 via crmd on essun.node2.com
    Stack: classic openais (with plugin)
    Current DC: essun.node1.com - partition with quorum
    Version: 1.1.10-14.el6_5.3-368c726
    2 Nodes configured, 2 expected votes
    5 Resources configured
    Node essun.node1.com: standby
    Online: [ essun.node2.com ]
     vip    (ocf::heartbeat:IPaddr):    Started essun.node2.com
     myserver   (lsb:mari): Started essun.node2.com
     Master/Slave Set: ms_dbdrdb [dbdrbd]
         Masters: [ essun.node2.com ]
         Stopped: [ essun.node1.com ]
     dbstore    (ocf::heartbeat:Filesystem):    Started essun.node2.com

    再次使用vip登录测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
[root@essun mariadb]# hostname
essun.node1.com
[root@essun mariadb]# /usr/local/mysql/bin/mysql -uroot -p -h192.168.1.150
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 9
Server version: 10.0.10-MariaDB-log MariaDB Server
Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+
4 rows in set (1.28 sec)










本文转自 jinlinger 51CTO博客，原文链接：http://blog.51cto.com/essun/1405326，如需转载请自行联系原作者

版权声明：本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。

网络协议 关系型数据库 MySQL Linux 开发工具
云数据库MariaDB版高可用
开发者社区 > 数据库 > 文章
数据库备份
作者高分内容
更多
Python:使用pycha快速绘制办公常用图(饼图、垂直直方图、水平直方图、散点图等七种图形) 3814 全面分析RHCE7（红帽认证工程师）考试题目之 ----Samba文件共享篇 3617 用Windows Server实现软件定义存储之存储空间直连 3386 linux配置Cacti 2463 yum 安装 出现GPG密钥失败 4260
相关文章
+关注
余二五
文章
问答
作者高分内容
更多
Python:使用pycha快速绘制办公常用图(饼图、垂直直方图、水平直方图、散点图等七种图形) 3814 全面分析RHCE7（红帽认证工程师）考试题目之 ----Samba文件共享篇 3617 用Windows Server实现软件定义存储之存储空间直连 3386 linux配置Cacti 2463 yum 安装 出现GPG密钥失败 4260
文章排行榜
最热
最新
1 承上启下：基于全域漏斗分析的主搜深度统一粗排 91555
2 复杂链表的复制 90660
3 【Transform3D】转换详解（看完就会） 88214
4 【flexbox弹性布局学习指南】CSS热门布局方案 68868
5 如何构建标签画像工程体系及实现方案 63378
6 DCM：中间件家族迎来新成员 61177
7 系统的混乱并业务本身之复杂，我们并不擅长处理『简单』 48289
8 超越10倍开发者 48162
9 ⚡Three.js-在场景中导入3D动画模型 46125
10 《JavaScript高级程序设计》__ 作用域&内存 37898
11 【从零开始学习深度学习】12. 什么是模型的训练误差？基于三阶多项式的欠拟合与过拟合训练过程演示 27906
12 记录下考Apsara Clouder云计算专项技能认证：云服务器ECS入门过程 20966
13 阿里云物联网平台设备分发实战 33512
14 ChatGPT 常见错误原因及解决方案：报错、回答不完整等 22533
15 学生速看！免费领取阿里云云服务器的详细攻略 4742
16 Python3，9行代码，对比两个Excel数据差异，并把差异结果重新保存。 8994
17 深聊性能测试，从入门到放弃之： Windows系统性能监控(一) 性能监视器介绍及使用。 9985
18 Python3，区区10行代码，批量把图片插入Excel指定单元格中，省下时间去烫头发。 3836
19 Python3，区区9行代码批量提取PDF文件的指定内容，我被震惊了.... 5682
20 Python3，os模块还可以这样玩，自动删除磁盘文件，非必要切勿操作。 11324
1 Java编程技巧之单元测试用例简化方法 62
2 语雀 117
3 如何选择离线数据集成方案-全量&增量 52
4 云时代，好用的数据迁移方案推荐 149
5 浅谈智能语音交互 127
6 Mysql深入优化（三） --- 查询缓存优化、内存管理及优化、MySQL锁问题 57
7 大数据知识面试题-Hive 53
8 【分享】宜搭集成自动化，Groovy节点一般用法，解析JSON数据 98
9 算法提升（二） 异或法 79
10 一封来自开发者社区的邀请函 183
11 【分享】在宜搭JS面板写了死循环如何解决 133
12 如何选择数据集成方式-离线&实时 151
13 网易游戏实时 HTAP 计费风控平台建设 222
14 Apache Tomcat 存在 JsonErrorReportValve 注入漏洞（CVE-2022-45143） 92
15 电影《流浪地球2》观后感 53
16 2023，不一样的数据库 117
17 公众号可订阅阿里云商标申请进度 51
18 Grid网格布局学习笔记 60
19 阿里云国际版创建ESC实例Linux系统 69
20 【数据设计与实现】第5章：同步与互斥 85
推荐文章
2022藏经阁年度电子书榜单出炉！ 参与应用引擎 SAE评测，赢取CHERRY机械键盘等好礼 “第益课”大学生技术公益 乘风者计划邀您入驻社区，精彩权益即刻享
相关电子书
更多
MySQL MGR 8.0高可用实战
立即下载
好的 MySQL 兼容可以做到什么程度
立即下载
深入MySQL实战
立即下载