CentOS7 多节点部署redis（redis4.0.9）集群

2018-06-01 4028

版权

本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。

本文涉及的产品

云数据库 Tair（兼容Redis），内存型 2GB

Redis 开源版，标准版 2GB

简介：

生产redis集群解决方案

参考链接：参考笔者之前写到一遍文章：
https://blog.csdn.net/wh211212/article/details/52817923

实验背景，业务上需要使用redis集群

环境CentOS7：

Hostname	Role	IP
redis1	redis:7000 7003	192.168.0.117
redis2	redis:7001 7004	192.168.0.118
redis3	redis:7002 7005	192.168.0.119

首先在三个redis节点上安装并启动至少6个redis实例

编译当前最新版本redis

安装依赖，CentOS7最小化安装

yum -y install epel-release && yum update -y
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

yum install vim net-tools -y

为了方便设置三台redis节点ssh免密，同步hosts如下：

# redis1为例：
# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain 

# redis cluster
192.168.0.117 redis1 ecs-117
192.168.0.118 redis2 ecs-118
192.168.0.119 redis3 ecs-119

redis1编译安装redis4.0.9

yum groupinstall "Development Tools" -y && yum install tcl wget -y

wget http://download.redis.io/releases/redis-4.0.9.tar.gz
tar zxvf redis-4.0.9.tar.gz -C /data/
ln -s /data/redis-4.0.9/ /data/redis 
cd /data/redis && make

# 创建redis常用命令目录
mkdir /data/redis/bin
cp /data/redis/src/{redis-benchmark,redis-check-aof,redis-check-rdb,redis-cli,redis-sentinel,redis-server,redis-shutdown,redis-trib.rb} /data/redis/bin/

笔者redis cluster目录结构，以redis1为例：

[root@redis1 data]# tree -d /data -L 1
/data
├── redis -> /data/redis-4.0.9/
├── redis-4.0.9
└── redis-cluster

# tree -d /data/redis-cluster/
/data/redis-cluster/
├── conf
├── nodes  # 存放redis 集群节点的配置文件
└── scripts # redis 集群维护脚本

# conf存放redis实例配置文件：7000.conf、7003.conf

脚本具体内容

[root@redis1 redis-cluster]# tree scripts/
scripts/
├── bgrewriteaof.sh
├── redis-7000.sh
└── redis-7003.sh

0 directories, 3 files
[root@redis1 redis-cluster]# cat scripts/bgrewriteaof.sh 
#!/bin/bash
#############################################
# Functions: start & stop redis cluster node
# ChangeLog:
# 2018-05-30 shaonbean@qq.com initial
############################################
# set -x #DEBUG

REDIS_BIN_DIR="/data/redis/bin"
REDIS_CLUSTER_DIR="/data/redis-cluster"
REDIS_CLI="$REDIS_BIN_DIR/redis-cli"
REDIS_SERVER="$REDIS_BIN_DIR/redis-server"

PORTLIST=(7000 7003)

bgrewriteaof() {

for REDIS_PORT in ${PORTLIST[@]};
    do
    REDIS_NAME="redis-$REDIS_PORT"
    REDIS_CONFIG="/data/redis-cluster/conf/$REDIS_PORT.conf"
    REDIS_PASS="Aniuredis123"
    echo -n $"Rewriteaof $REDIS_NAME: "
    $REDIS_CLI -p $REDIS_PORT -a $REDIS_PASS BGREWRITEAOF
    retval=$?
    [ $retval -eq 0 ] && echo " $REDIS_NAME bgrewriteaof succeed!"  
    done
}

bgrewriteaof  

# 定时进行aof重写

[root@redis1 redis-cluster]# cat scripts/redis-7000.sh # redis实例启动脚本
#!/bin/bash
#############################################
# Functions: start & stop redis cluster node
# ChangeLog:
# 2018-05-30 shaonbean@qq.com initial
############################################
# set -x #DEBUG

REDIS_BIN_DIR="/data/redis/bin"
REDIS_CLUSTER_DIR="/data/redis-cluster"
REDIS_CLI="$REDIS_BIN_DIR/redis-cli"
REDIS_SERVER="$REDIS_BIN_DIR/redis-server"

REDIS_USER="root"
REDIS_PORT=7000
REDIS_NAME="redis-$REDIS_PORT"
REDIS_CONFIG="/data/redis-cluster/conf/$REDIS_PORT.conf"
REDIS_PASS="Aniuredis123"

start() {
    [ -f $REDIS_CONFIG ] || exit 6
    [ -x $REDIS_SERVER ] || exit 5
    echo -n $"Starting $REDIS_NAME: "
    $REDIS_SERVER $REDIS_CONFIG
    retval=$?
    [ $retval -eq 0 ] && echo " $REDIS_NAME start succeed!"
}

stop() {
    echo -n $"Stopping $REDIS_NAME: "
    $REDIS_CLI -p $REDIS_PORT -a $REDIS_PASS shutdown
    retval=$?
    [ $retval -eq 0 ] && echo " $REDIS_NAME stop succeed!"    
}

restart() {
    stop
    start
}

case "$1" in
    start)
        $1
        ;;
    stop)
        $1
        ;;
    restart)
        $1
        ;;
    *)
        echo $"Usage: $0 {start|stop|restart}"
        exit 2
esac
exit $?

7000.conf

[root@redis1 conf]# cat 7000.conf 
bind 0.0.0.0
protected-mode yes
port 7000
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /var/run/redis/redis_7000.pid
loglevel notice
logfile /var/log/redis/redis-7000.log
databases 16
always-show-logo yes
save ""
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump_7000.rdb
dir /var/lib/redis
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
masterauth Aniuredis123
requirepass Aniuredis123
rename-command FLUSHALL ""
#rename-command CONFIG ""
maxclients 10000
maxmemory 4gb
maxmemory-policy allkeys-lru
maxmemory-samples 5
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
slave-lazy-flush no
appendonly yes
appendfilename "appendonly_7000.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble no
lua-time-limit 5000
# cluster config
cluster-enabled yes
cluster-config-file /data/redis-cluster/nodes/nodes-7000.conf
cluster-node-timeout 5000
cluster-slave-validity-factor 10
cluster-migration-barrier 1
cluster-require-full-coverage yes
cluster-slave-no-failover no
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes

配置设置

安装ruby环境

yum --enablerepo=centos-sclo-rh -y install rh-ruby24
scl enable rh-ruby24 bash # 启用ruby环境变量

# 写入到环境变量
vi /etc/profile.d/rh-ruby24.sh
#!/bin/bash
source /opt/rh/rh-ruby24/enable
export X_SCLS="`scl enable rh-ruby24 'echo $X_SCLS'`"

创建集群安装依赖

# gem install redis
Successfully installed redis-4.0.1
Parsing documentation for redis-4.0.1
Done installing documentation for redis after 2 seconds
1 gem installed

以redis1为例：

[root@redis1 ~]# /data/redis/bin/redis-trib.rb create --replicas 1 192.168.0.117:7000 192.168.0.118:7001 192.168.0.119:7002 192.168.0.117:7003 192.168.0.118:7004 192.168.0.119:7005
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.0.117:7000
192.168.0.118:7001
192.168.0.119:7002
Adding replica 192.168.0.118:7004 to 192.168.0.117:7000
Adding replica 192.168.0.119:7005 to 192.168.0.118:7001
Adding replica 192.168.0.117:7003 to 192.168.0.119:7002
M: 1ea6b06582a23a1c73d06bc1aa32c3f4d8edcb24 192.168.0.117:7000
   slots:0-5460 (5461 slots) master
M: 2da83fad766d22198f064b31524d7e757af63a04 192.168.0.118:7001
   slots:5461-10922 (5462 slots) master
M: 77aa1ba8353e1dc303c1f3f2a553f82777aad5d6 192.168.0.119:7002
   slots:10923-16383 (5461 slots) master
S: 76feb9dfb7963a740e697bc9ecbdf4d18d1cdbb4 192.168.0.117:7003
   replicates 77aa1ba8353e1dc303c1f3f2a553f82777aad5d6
S: 769eebd4bde8bbc58543585021bc34059d50ba23 192.168.0.118:7004
   replicates 1ea6b06582a23a1c73d06bc1aa32c3f4d8edcb24
S: 6f2b369304597814af228b32c9c384581a34b900 192.168.0.119:7005
   replicates 2da83fad766d22198f064b31524d7e757af63a04
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 192.168.0.117:7000)
M: 1ea6b06582a23a1c73d06bc1aa32c3f4d8edcb24 192.168.0.117:7000
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 76feb9dfb7963a740e697bc9ecbdf4d18d1cdbb4 192.168.0.117:7003
   slots: (0 slots) slave
   replicates 77aa1ba8353e1dc303c1f3f2a553f82777aad5d6
M: 2da83fad766d22198f064b31524d7e757af63a04 192.168.0.118:7001
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: 77aa1ba8353e1dc303c1f3f2a553f82777aad5d6 192.168.0.119:7002
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 769eebd4bde8bbc58543585021bc34059d50ba23 192.168.0.118:7004
   slots: (0 slots) slave
   replicates 1ea6b06582a23a1c73d06bc1aa32c3f4d8edcb24
S: 6f2b369304597814af228b32c9c384581a34b900 192.168.0.119:7005
   slots: (0 slots) slave
   replicates 2da83fad766d22198f064b31524d7e757af63a04
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

redis集群使用密码设置

配置文件写入：

masterauth Aniuredis123 
requirepass Aniuredis123

2、设置密码之后如果需要使用redis-trib.rb的各种命令,报错解决：

# 解决办法：
[root@redis1 redis]# find / -name client.rb
/opt/rh/rh-ruby24/root/usr/local/share/gems/gems/redis-4.0.1/lib/redis/client.rb

class Client
    DEFAULTS = {
      :url => lambda { ENV["REDIS_URL"] },
      :scheme => "redis",
      :host => "127.0.0.1",
      :port => 6379,
      :path => nil,
      :timeout => 5.0,
      :password => "passwd123",
      :db => 0,
      :driver => nil,
      :id => nil,
      :tcp_keepalive => 0,
      :reconnect_attempts => 1,
      :inherit_socket => false
    }

注意：client.rb路径可以通过find命令查找：find / -name 'client.rb'

笔者redis 集群安装总结

# redis集群说明

- redis1是Redis集群的一个节点A，上面运行两个redis实例，7000 7003
- redis2是Redis集群的一个节点B，上面运行两个redis实例，7001 7004
- redis3是Redis集群的一个节点C，上面运行两个redis实例，7002 7005


- 假设集群包含A、B、C、A1、B1、C1六个节点

A、B、C为主节点对应Redis实例：7000 7001 7002
A1、B1、C1为从节点对应redis实例：7003 7004 7005

# 建议交叉设置主从节点，对应关系为
A > B1
B > C1
C > A1

# 创建集群
cd /data/redis/bin
./redis-trib.rb create --replicas 1 192.168.0.117:7000 192.168.0.118:7001 192.168.0.119:7002 192.168.0.117:7003 192.168.0.118:7004 192.168.0.119:7005

# redis集群性能测试
time /usr/local/redis/bin/redis-benchmark -h redis1 -p 7000 -a Aniuredis123 -c 200 -r 1000000 -n 2000000 -t get,set,lpush,lpop -P 16 -q
time /usr/local/redis/bin/redis-benchmark -h redis2 -p 7001 -a Aniuredis123 -c 200 -r 1000000 -n 2000000 -t get,set,lpush,lpop -P 16 -q
time /usr/local/redis/bin/redis-benchmark -h redis3 -p 7002 -a Aniuredis123 -c 200 -r 1000000 -n 2000000 -t get,set,lpush,lpop -P 16 -q

# redis故障模拟


# 参考redis.service设置redis实例开机自启动
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo 511 > /proc/sys/net/core/somaxconn

建议参考笔者的目录结构去部署，快下班啦，后面有需要注意的问题在总结

redis 集群结合F5提供服务

这里写图片描述

CentOS7 多节点部署redis（redis4.0.9）集群

生产redis集群解决方案

编译当前最新版本redis

配置设置

redis集群使用密码设置

笔者redis 集群安装总结

redis 集群结合F5提供服务

热门文章

最新文章

相关课程

相关电子书

相关实验场景

推荐镜像

热门

活动广场

任务中心

开发者评测

高校计划

乘风者计划

训练营

阿里云MVP

话题

直播

下载

镜像站

技术资料

插件

CentOS7 多节点部署redis（redis4.0.9）集群

生产redis集群解决方案

编译当前最新版本redis

配置设置

redis集群使用密码设置

笔者redis 集群安装总结

redis 集群结合F5提供服务

热门文章

最新文章

相关课程

相关电子书

相关实验场景

推荐镜像