Redis Cluster 搭建手册

1 下载最新版redis

wget 'http://download.redis.io/releases/redis-3.2.3.tar.gz'

2  编译安装

#安装到/opt/redis-cluster目录

tar zxvf redis-3.2.3.tar.gz

cd redis-3.2.3/

make

make PREFIX=/opt/redis-cluster install

#创建配置目录，拷贝相关文件

mkdir /opt/redis-cluster/{log conf rdb run}

#redis目录

cp src/redis-trib.rb /opt/redis-cluster/bin/

3  安装其他依赖包

apt-get install ruby 2.3.1

gem install redis

4  创建配置文件

redis.conf

bind 0.0.0.0

daemonize yes

pidfile /opt/redis-cluster/run/redis-6379.pid

（pidfile /usr/local/etc/redis/run/redis.pid）

port 6379

tcp-backlog 511

timeout 0

tcp-keepalive 0

loglevel notice

logfile "/opt/redis-cluster/log/redis-6379.log"

databases 16

save 900 1

save 300 10

save 60 10000

stop-writes-on-bgsave-error yes

rdbcompression yes

rdbchecksum yes

dbfilename dump-6379.rdb

dir /opt/redis-cluster/rdb

slave-serve-stale-data yes

slave-read-only yes

repl-diskless-sync no

repl-diskless-sync-delay 5

repl-disable-tcp-nodelay no

slave-priority 100

appendonly yes

appendfilename "redis-6379.aof"

appendfsync everysec

no-appendfsync-on-rewrite no

auto-aof-rewrite-percentage 100

auto-aof-rewrite-min-size 64mb

aof-load-truncated yes

lua-time-limit 5000

cluster-enabled yes

cluster-config-file nodes-6379.conf

cluster-node-timeout 15000

slowlog-log-slower-than 10000

slowlog-max-len 128

latency-monitor-threshold 0

notify-keyspace-events ""

hash-max-ziplist-entries 512

hash-max-ziplist-value 64

list-max-ziplist-entries 512

list-max-ziplist-value 64

set-max-intset-entries 512

zset-max-ziplist-entries 128

zset-max-ziplist-value 64

hll-sparse-max-bytes 3000

activerehashing yes

client-output-buffer-limit normal 0 0 0

client-output-buffer-limit slave 256mb 64mb 60

client-output-buffer-limit pubsub 32mb 8mb 60

hz 10

aof-rewrite-incremental-fsync yes

其实大部分都是原有的配置信息，不需要做大的改动，红色文字部分都需要调整，与集群相关的配置如下：

cluster-enabled yes

cluster-config-file nodes-6379.conf

cluster-node-timeout 15000

注意其中的nodes-6379.conf这个文件不需要创建，在初始化集群的时候会自动创建的。

5    在启动redis实例前先修改一些系统级别的配置

echo never > /sys/kernel/mm/transparent_hugepage/enabled

# 打开/etc/sysctl.conf，追加如下内容

vm.overcommit_memory = 1

#使配置生效

sysctl -p

6    启动所有redis实例

cd /opt/redis-cluster/bin

./redis-server /opt/redis-cluster/conf/redis-6379.conf

./redis-server /opt/redis-cluster/conf/redis-6380.conf

./redis-server /opt/redis-cluster/conf/redis-6381.conf

./redis-server /opt/redis-cluster/conf/redis-7379.conf

./redis-server /opt/redis-cluster/conf/redis-7380.conf

./redis-server /opt/redis-cluster/conf/redis-7381.conf

检查一下端口监听的状态：

netstat -tupln

7    redis-trib.rb

        redis-trib.rb是一个官方提供的用来操作cluster的ruby脚本，我们后面管理cluster会经常使用到这个脚本

Usage: redis-trib <command> <options> <arguments ...>

  create          host1:port1 ... hostN:portN

                  --replicas <arg>

  check           host:port

  fix             host:port

  reshard         host:port

                  --from <arg>

                  --to <arg>

                  --slots <arg>

                  --yes

  add-node        new_host:new_port existing_host:existing_port

                  --slave

                  --master-id <arg>

  del-node        host:port node_id

  set-timeout     host:port milliseconds

  call            host:port command arg arg .. arg

  import          host:port

                  --from <arg>

  help            (show this help)

For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.

8    初始化启动cluster

切记用具体IP启动

./redis-trib.rb create --replicas 1 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381 127.0.0.1:7379 127.0.0.1:7380 127.0.0.1:7381

./redis-trib.rb create --replicas 1 10.157.26.84:6379 10.157.26.85:6379 10.157.26.86:6379 10.157.26.87:6379 10.157.26.88:6379 10.157.26.89:6379

创建结果如下

>>> Creating cluster

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

127.0.0.1:6379

127.0.0.1:6380

127.0.0.1:6381

Adding replica 127.0.0.1:7379 to 127.0.0.1:6379

Adding replica 127.0.0.1:7380 to 127.0.0.1:6380

Adding replica 127.0.0.1:7381 to 127.0.0.1:6381

M: e85567eff146a649f971b7e543e8421fe64c29e0 127.0.0.1:6379

   slots:0-5460 (5461 slots) master

M: b58997471998a59a4238bec506a8452a04187749 127.0.0.1:6380

   slots:5461-10922 (5462 slots) master

M: 34c068fe3182321bb53b5c3a106676095dc03bdd 127.0.0.1:6381

   slots:10923-16383 (5461 slots) master

S: bc4f4a6c25135dc9ede0428b3567d4e5a6db87bf 127.0.0.1:7379

   replicates e85567eff146a649f971b7e543e8421fe64c29e0

S: ea56cb0020e8c24a55c0301bd073aedc99da0206 127.0.0.1:7380

   replicates b58997471998a59a4238bec506a8452a04187749

S: 021ea707f99795088d474cc622b25b928e8ed73f 127.0.0.1:7381

   replicates 34c068fe3182321bb53b5c3a106676095dc03bdd

Can I set the above configuration? (type 'yes' to accept): yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join...

>>> Performing Cluster Check (using node 127.0.0.1:6379)

M: e85567eff146a649f971b7e543e8421fe64c29e0 127.0.0.1:6379

   slots:0-5460 (5461 slots) master

M: b58997471998a59a4238bec506a8452a04187749 127.0.0.1:6380

   slots:5461-10922 (5462 slots) master

M: 34c068fe3182321bb53b5c3a106676095dc03bdd 127.0.0.1:6381

   slots:10923-16383 (5461 slots) master

M: bc4f4a6c25135dc9ede0428b3567d4e5a6db87bf 127.0.0.1:7379

   slots: (0 slots) master

   replicates e85567eff146a649f971b7e543e8421fe64c29e0

M: ea56cb0020e8c24a55c0301bd073aedc99da0206 127.0.0.1:7380

   slots: (0 slots) master

   replicates b58997471998a59a4238bec506a8452a04187749

M: 021ea707f99795088d474cc622b25b928e8ed73f 127.0.0.1:7381

   slots: (0 slots) master

   replicates 34c068fe3182321bb53b5c3a106676095dc03bdd

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

  检查一下cluster的状态： ./redis-trib.rb check 127.0.0.1:6379

>>> Performing Cluster Check (using node 127.0.0.1:6379)

M: e85567eff146a649f971b7e543e8421fe64c29e0 127.0.0.1:6379

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

S: ea56cb0020e8c24a55c0301bd073aedc99da0206 127.0.0.1:7380

   slots: (0 slots) slave

   replicates b58997471998a59a4238bec506a8452a04187749

S: bc4f4a6c25135dc9ede0428b3567d4e5a6db87bf 127.0.0.1:7379

   slots: (0 slots) slave

   replicates e85567eff146a649f971b7e543e8421fe64c29e0

M: 34c068fe3182321bb53b5c3a106676095dc03bdd 127.0.0.1:6381

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 021ea707f99795088d474cc622b25b928e8ed73f 127.0.0.1:7381

   slots: (0 slots) slave

   replicates 34c068fe3182321bb53b5c3a106676095dc03bdd

M: b58997471998a59a4238bec506a8452a04187749 127.0.0.1:6380

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

9  随便接入cluster的某个节点进行测试

redis-cli -c -h 127.0.0.1 -p 6379

注意：redis-cli在接入集群模式时，要使用-c参数  

127.0.0.1:6379> keys *

(empty list or set)

127.0.0.1:6379> set test nidaye

-> Redirected to slot [6918] located at 127.0.0.1:6380

OK

127.0.0.1:6380> get test

"nidaye"

127.0.0.1:6380> set nihaogepi nihaogepi

-> Redirected to slot [5243] located at 127.0.0.1:6379

OK

127.0.0.1:6379>

redis.conf配置文件