休息了一段时间后，技术忘记的差不多了，现在公司搞了一个MariaDB集群用docker跑的，我自己用三台docker容器尝试了一下搭建过程。docker和MariaDB这里不作太多的介绍，想了解的朋友可以去官网查看。
环境信息如下：
IP地址 主机名 系统版本
172.17.0.18 test-node1 CentOS Linux release 7.2.1511 (Core)
172.17.0.19 test-node2 CentOS Linux release 7.2.1511 (Core)
172.17.0.21 test-node3 CentOS Linux release 7.2.1511 (Core)
关闭三台机器的selinux和防火墙哈~~~
1、三台机器都添加mariadb的源，内容如下：
# MariaDB 10.1 CentOS repository list - created 2015-11-30 13:13 UTC
# http://mariadb.org/mariadb/repositories/
[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/10.1/centos7-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1
2、三台机器都安装mariadb相关的软件包
yum -y install MariaDB-server MariaDB-client galera ===>最后一个是集群需要的软件
3、安装集群之间同步的软件，这里我们推荐使用percona公司的xtrabackup工具，因为官方推荐也是这种，当然默认是rsync工具。
yum -y install http://www.percona.com/downloads/percona-release/redhat/0.1-3/percona-release-0.1-3.noarch.rpm
yum -y percona-xtrabackup-24 which socat ===>最后两个软件包是同步时候需要使用的命令，不然第二和第三个数据库会启动失败。
相关解释文档如下：
4、初始化第一个数据库及修改属主属组
mysql_install_db
chown -R mysql.mysql /var/lib/mysql
5、修改第一个启动数据库的server.cnf文件，默认配置文件如下：
more /etc/my.cnf.d/server.cnf
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql/
#
# this is read by the standalone daemon and embedded servers
[server]
# this is only for the mysqld standalone daemon
[mysqld]
#
# * Galera-related settings
#
[galera]
# Mandatory settings
#wsrep_on=ON
#wsrep_provider=
#wsrep_cluster_address=
#binlog_format=row
#default_storage_engine=InnoDB
#innodb_autoinc_lock_mode=2
#
# Allow server to accept connections on all interfaces.
#
#bind-address=0.0.0.0
#
# Optional setting
#wsrep_slave_threads=1
#innodb_flush_log_at_trx_commit=0
# this is only for embedded server
[embedded]
# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]
# This group is only read by MariaDB-10.1 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.1]
修改为如下内容：
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql/
#
# this is read by the standalone daemon and embedded servers
[server]
# this is only for the mysqld standalone daemon
[mysqld]
# GENERAL #
user = mysql
# DATA STORAGE #
datadir = /var/lib/mysql
#
# * Galera-related settings
#
[galera]
wsrep_on = on
wsrep_sst_auth = test-cluster:test-password
wsrep_sst_method = xtrabackup-v2
wsrep_cluster_name = test-cluster
wsrep_cluster_address = gcomm:// ===>注意这里一个都不写
wsrep_node_name = test-node1
wsrep_node_address = 172.17.0.18:4567
wsrep_provider = /usr/lib64/galera/libgalera_smm.so
#wsrep_slave_threads = 8
#innodb-flush-log-at-trx-commit = 2
#
# Allow server to accept connections on all interfaces.
#
#bind-address=0.0.0.0
#
# Optional setting
#wsrep_slave_threads=1
#innodb_flush_log_at_trx_commit=0
# BINARY LOGGING #
binlog-format = row
# this is only for embedded server
[embedded]
# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]
# This group is only read by MariaDB-10.1 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.1]
#
# These groups are read by MariaDB command-line tools
# Use it for options that affect only one utility
#
6、启动第一个数据库，启动方式如下：
nohup mysqld --wsrep-new-cluster &
7、登录第一个数据库授权同步用户和密码，注意用户和密码和上面配置文件里面定义的相同
grant all on *.* to 'test-cluster'@'localhost' identified by 'test-password';
8、分别修改第二个和第三个数据库的server.cnf文件，内容主要修改如下：
wsrep_on = on
wsrep_sst_auth = test-cluster:test-password
wsrep_sst_method = xtrabackup-v2
wsrep_cluster_name = test-cluster
wsrep_cluster_address = gcomm://172.17.0.18:4567,172.17.0.19:4567 ===>注意这里写第一台数据库和本身
wsrep_node_name = test-node2
wsrep_node_address = 172.17.0.19:4567
wsrep_provider = /usr/lib64/galera/libgalera_smm.so
启动第二台数据库
/etc/init.d/mysql start ===>记得观察输出日志
===================下面是第三台数据库server.cnf的内容======================
wsrep_on = on
wsrep_sst_auth = test-cluster:test-password
wsrep_sst_method = xtrabackup-v2
wsrep_cluster_name = test-cluster
wsrep_cluster_address = gcomm://172.17.0.18:4567,172.17.0.19:4567,172.17.0.21:4567 ===>写集群中所有的
wsrep_node_name = test-node3
wsrep_node_address = 172.17.0.21:4567
wsrep_provider = /usr/lib64/galera/libgalera_smm.so
同样启动第三台数据库
/etc/init.d/mysql start ===>记得观察输出日志
如果正常按照我的步骤操作的话应该启动成功，剩下的就是测试了。
登录第二台数据库查看
登录第三台数据库查看
上面显示正常同步了，注意一般生产环境不要轻易重启第一个数据库，因为重启第一个数据库后它的数据会和别的节点的数据不同步，这样就发生了脑裂。虽然是多主模式，但是还是建议只往一个数据库上写，从其余所有的数据库来读，记得重启了第一个数据库，其它数据库也要重启，而且是一台重启数据同步完成后再重启下一台，这样很费时间和IO。另外生产环境注意MariaDB集群都接入千兆交换机下面。不然性能上不去。
我上面server.cnf配置文件基本都是最简单的写法，下面附一个生产环境的写法。
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql/
#
# this is read by the standalone daemon and embedded servers
[server]
# this is only for the mysqld standalone daemon
[mysqld]
# GENERAL #
user = mysql
default-storage-engine = InnoDB
socket = /var/lib/mysql/mysql.sock
pid-file = /var/lib/mysql/mysql.pid
bind-address = 0.0.0.0
# CHARACTER SET #
collation-server = utf8_unicode_ci
init-connect = 'SET NAMES utf8'
character-set-server = utf8
# MyISAM #
key-buffer-size = 32M
myisam-recover-options = FORCE,BACKUP
# SAFETY #
skip-host-cache
skip-name-resolve
max-allowed-packet = 16M
max-connect-errors = 1000000
#sql-mode = STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_AUTO_VALUE_ON_ZERO,NO_ENGINE_SUBSTITUTION,NO_ZERO_DATE,NO_ZERO_IN_DATE
sysdate-is-now = 1
innodb = FORCE
innodb-strict-mode = 1
innodb-autoinc-lock-mode = 2
innodb-doublewrite = 1
innodb_flush_log_at_trx_commit = 0
innodb_file_per_table = 1
# DATA STORAGE #
datadir = /var/lib/mysql
# BINARY LOGGING #
log-bin = /var/lib/mysql/mysql-bin
expire-logs-days = 2
sync-binlog = 1
binlog-format = row
# CACHES AND LIMITS #
tmp-table-size = 32M
max-heap-table-size = 32M
query-cache-type = 0
query-cache-size = 0
max-connections = 500
thread-cache-size = 50
open-files-limit = 65535
table-definition-cache = 4096
table-open-cache = 4096
# INNODB #
innodb-flush-method = O_DIRECT
innodb-log-files-in-group = 2
innodb-log-file-size = 128M
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table = 1
innodb-buffer-pool-size = 128M
# LOGGING #
log-error = /dev/stdout
slow-query-log-file = /var/lib/mysql
log-queries-not-using-indexes = 1
slow-query-log = 1
#
# * Galera-related settings
#
[galera]
wsrep_provider = /usr/lib64/galera/libgalera_smm.so
wsrep_slave_threads = 8
innodb-flush-log-at-trx-commit = 2
# this is only for embedded server
[embedded]
# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]
# This group is only read by MariaDB-10.1 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.1]
#
# These groups are read by MariaDB command-line tools
# Use it for options that affect only one utility
#
[mysql]
port = 3306
socket = /var/lib/mysql/mysql.sock
上述里没有集群配置信息，我们可以在/etc/my.cnf.d/galera.cnf文件里面写入集群相关信息。
[galera]
wsrep_on = on
wsrep_sst_auth = test-cluster:test-password
wsrep_sst_method = xtrabackup-v2
wsrep_cluster_name = test-node
wsrep_cluster_address = gcomm://xxxx:4567,xxxx:4567,xxxx:4567
wsrep_node_name = test-nodex
wsrep_node_address = 本机IP地址:4567
环境信息如下：
IP地址 主机名 系统版本
172.17.0.18 test-node1 CentOS Linux release 7.2.1511 (Core)
172.17.0.19 test-node2 CentOS Linux release 7.2.1511 (Core)
172.17.0.21 test-node3 CentOS Linux release 7.2.1511 (Core)
关闭三台机器的selinux和防火墙哈~~~
1、三台机器都添加mariadb的源，内容如下：
# MariaDB 10.1 CentOS repository list - created 2015-11-30 13:13 UTC
# http://mariadb.org/mariadb/repositories/
[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/10.1/centos7-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1
2、三台机器都安装mariadb相关的软件包
yum -y install MariaDB-server MariaDB-client galera ===>最后一个是集群需要的软件
3、安装集群之间同步的软件，这里我们推荐使用percona公司的xtrabackup工具，因为官方推荐也是这种，当然默认是rsync工具。
yum -y install http://www.percona.com/downloads/percona-release/redhat/0.1-3/percona-release-0.1-3.noarch.rpm
yum -y percona-xtrabackup-24 which socat ===>最后两个软件包是同步时候需要使用的命令，不然第二和第三个数据库会启动失败。
相关解释文档如下：
4、初始化第一个数据库及修改属主属组
mysql_install_db
chown -R mysql.mysql /var/lib/mysql
5、修改第一个启动数据库的server.cnf文件，默认配置文件如下：
more /etc/my.cnf.d/server.cnf
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql/
#
# this is read by the standalone daemon and embedded servers
[server]
# this is only for the mysqld standalone daemon
[mysqld]
#
# * Galera-related settings
#
[galera]
# Mandatory settings
#wsrep_on=ON
#wsrep_provider=
#wsrep_cluster_address=
#binlog_format=row
#default_storage_engine=InnoDB
#innodb_autoinc_lock_mode=2
#
# Allow server to accept connections on all interfaces.
#
#bind-address=0.0.0.0
#
# Optional setting
#wsrep_slave_threads=1
#innodb_flush_log_at_trx_commit=0
# this is only for embedded server
[embedded]
# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]
# This group is only read by MariaDB-10.1 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.1]
修改为如下内容：
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql/
#
# this is read by the standalone daemon and embedded servers
[server]
# this is only for the mysqld standalone daemon
[mysqld]
# GENERAL #
user = mysql
# DATA STORAGE #
datadir = /var/lib/mysql
#
# * Galera-related settings
#
[galera]
wsrep_on = on
wsrep_sst_auth = test-cluster:test-password
wsrep_sst_method = xtrabackup-v2
wsrep_cluster_name = test-cluster
wsrep_cluster_address = gcomm:// ===>注意这里一个都不写
wsrep_node_name = test-node1
wsrep_node_address = 172.17.0.18:4567
wsrep_provider = /usr/lib64/galera/libgalera_smm.so
#wsrep_slave_threads = 8
#innodb-flush-log-at-trx-commit = 2
#
# Allow server to accept connections on all interfaces.
#
#bind-address=0.0.0.0
#
# Optional setting
#wsrep_slave_threads=1
#innodb_flush_log_at_trx_commit=0
# BINARY LOGGING #
binlog-format = row
# this is only for embedded server
[embedded]
# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]
# This group is only read by MariaDB-10.1 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.1]
#
# These groups are read by MariaDB command-line tools
# Use it for options that affect only one utility
#
6、启动第一个数据库，启动方式如下：
nohup mysqld --wsrep-new-cluster &
7、登录第一个数据库授权同步用户和密码，注意用户和密码和上面配置文件里面定义的相同
grant all on *.* to 'test-cluster'@'localhost' identified by 'test-password';
8、分别修改第二个和第三个数据库的server.cnf文件，内容主要修改如下：
wsrep_on = on
wsrep_sst_auth = test-cluster:test-password
wsrep_sst_method = xtrabackup-v2
wsrep_cluster_name = test-cluster
wsrep_cluster_address = gcomm://172.17.0.18:4567,172.17.0.19:4567 ===>注意这里写第一台数据库和本身
wsrep_node_name = test-node2
wsrep_node_address = 172.17.0.19:4567
wsrep_provider = /usr/lib64/galera/libgalera_smm.so
启动第二台数据库
/etc/init.d/mysql start ===>记得观察输出日志
===================下面是第三台数据库server.cnf的内容======================
wsrep_on = on
wsrep_sst_auth = test-cluster:test-password
wsrep_sst_method = xtrabackup-v2
wsrep_cluster_name = test-cluster
wsrep_cluster_address = gcomm://172.17.0.18:4567,172.17.0.19:4567,172.17.0.21:4567 ===>写集群中所有的
wsrep_node_name = test-node3
wsrep_node_address = 172.17.0.21:4567
wsrep_provider = /usr/lib64/galera/libgalera_smm.so
同样启动第三台数据库
/etc/init.d/mysql start ===>记得观察输出日志
如果正常按照我的步骤操作的话应该启动成功，剩下的就是测试了。
登录第二台数据库查看
登录第三台数据库查看
上面显示正常同步了，注意一般生产环境不要轻易重启第一个数据库，因为重启第一个数据库后它的数据会和别的节点的数据不同步，这样就发生了脑裂。虽然是多主模式，但是还是建议只往一个数据库上写，从其余所有的数据库来读，记得重启了第一个数据库，其它数据库也要重启，而且是一台重启数据同步完成后再重启下一台，这样很费时间和IO。另外生产环境注意MariaDB集群都接入千兆交换机下面。不然性能上不去。
我上面server.cnf配置文件基本都是最简单的写法，下面附一个生产环境的写法。
#
# These groups are read by MariaDB server.
# Use it for options that only the server (but not clients) should see
#
# See the examples of server my.cnf files in /usr/share/mysql/
#
# this is read by the standalone daemon and embedded servers
[server]
# this is only for the mysqld standalone daemon
[mysqld]
# GENERAL #
user = mysql
default-storage-engine = InnoDB
socket = /var/lib/mysql/mysql.sock
pid-file = /var/lib/mysql/mysql.pid
bind-address = 0.0.0.0
# CHARACTER SET #
collation-server = utf8_unicode_ci
init-connect = 'SET NAMES utf8'
character-set-server = utf8
# MyISAM #
key-buffer-size = 32M
myisam-recover-options = FORCE,BACKUP
# SAFETY #
skip-host-cache
skip-name-resolve
max-allowed-packet = 16M
max-connect-errors = 1000000
#sql-mode = STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_AUTO_VALUE_ON_ZERO,NO_ENGINE_SUBSTITUTION,NO_ZERO_DATE,NO_ZERO_IN_DATE
sysdate-is-now = 1
innodb = FORCE
innodb-strict-mode = 1
innodb-autoinc-lock-mode = 2
innodb-doublewrite = 1
innodb_flush_log_at_trx_commit = 0
innodb_file_per_table = 1
# DATA STORAGE #
datadir = /var/lib/mysql
# BINARY LOGGING #
log-bin = /var/lib/mysql/mysql-bin
expire-logs-days = 2
sync-binlog = 1
binlog-format = row
# CACHES AND LIMITS #
tmp-table-size = 32M
max-heap-table-size = 32M
query-cache-type = 0
query-cache-size = 0
max-connections = 500
thread-cache-size = 50
open-files-limit = 65535
table-definition-cache = 4096
table-open-cache = 4096
# INNODB #
innodb-flush-method = O_DIRECT
innodb-log-files-in-group = 2
innodb-log-file-size = 128M
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table = 1
innodb-buffer-pool-size = 128M
# LOGGING #
log-error = /dev/stdout
slow-query-log-file = /var/lib/mysql
log-queries-not-using-indexes = 1
slow-query-log = 1
#
# * Galera-related settings
#
[galera]
wsrep_provider = /usr/lib64/galera/libgalera_smm.so
wsrep_slave_threads = 8
innodb-flush-log-at-trx-commit = 2
# this is only for embedded server
[embedded]
# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]
# This group is only read by MariaDB-10.1 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.1]
#
# These groups are read by MariaDB command-line tools
# Use it for options that affect only one utility
#
[mysql]
port = 3306
socket = /var/lib/mysql/mysql.sock
上述里没有集群配置信息，我们可以在/etc/my.cnf.d/galera.cnf文件里面写入集群相关信息。
[galera]
wsrep_on = on
wsrep_sst_auth = test-cluster:test-password
wsrep_sst_method = xtrabackup-v2
wsrep_cluster_name = test-node
wsrep_cluster_address = gcomm://xxxx:4567,xxxx:4567,xxxx:4567
wsrep_node_name = test-nodex
wsrep_node_address = 本机IP地址:4567
