redis cluster在设计的时候,就考虑到了去中心化,去中间件,也就是说,集群中的每个节点都是平等的关系,都是对等的,每个节点都保存各自的数据和整个集群的状态。每个节点都和其他所有节点连接,而且这些连接保持活跃,这样就保证了我们只需要连接集群中的任意一个节点,就可以获取到其他节点的数据。
Redis 集群没有并使用传统的一致性哈希来分配数据,而是采用另外一种叫做哈希槽 (hash slot)
的方式来分配的。redis cluster 默认分配了 16384 个slot,当我们set一个key 时,会用CRC16
算法来取模得到所属的slot
,然后将这个key 分到哈希槽区间的节点上,具体算法就是:CRC16(key) % 16384。所以我们在测试的时候看到set 和 get 的时候,直接跳转到了7000端口的节点。
Redis 集群会把数据存在一个 master 节点,然后在这个 master 和其对应的salve 之间进行数据同步。当读取数据时,也根据一致性哈希算法到对应的 master 节点获取数据。只有当一个master 挂掉之后,才会启动一个对应的 salve 节点,充当 master 。
需要注意的是:必须要3个或以上
的主节点,否则在创建集群时会失败,并且当存活的主节点数小于总节点数的一半时,整个集群就无法提供服务了。
用两台虚拟机模拟6个节点,一台机器3个节点,创建出3 master、3 salve 环境
版本要求: redis 3.2.2
要让集群正常工作至少需要3个主节点,在这里我们要创建6个redis节点,其中三个为主节点,三个为从节点,对应的redis节点的ip和端口对应关系如下:
192.168.1.53:6379
192.168.1.53:6380
192.168.1.53:6381
192.168.1.53:6382
192.168.1.53:6383
192.168.1.53:6384
下载redis, 3.2.2 安装 http://download.redis.io/releases/redis-3.2.2.tar.gz
安装步骤请参考:如何在Linux下Redis安装
创建目录:
[root@localhost redis]# pwd /data/redis [root@localhost redis]# ls 6379 6380 6381 6382 6383 6384 [root@localhost redis]# cd 6379 [root@localhost 6379]# ls 6379.aof 6379.conf 6379.rdb nodes-6379.conf
修改配置文件redis.conf
daemonize yes maxmemory 4gb maxmemory-policy volatile-lru pidfile "/var/run/6380.pid" port 6380 bind 0.0.0.0 tcp-backlog 4096 timeout 0 tcp-keepalive 0 loglevel notice logfile "/tmp/6380.log" databases 16 #save 900 1 #save 300 10 save 60 2000 save 300 10000 save 900 50000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename "6380.rdb" dir "/data/redis/6380" slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 appendonly yes appendfilename "6380.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes lua-time-limit 5000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-entries 512 list-max-ziplist-value 64 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes cluster-enabled yes cluster-config-file "nodes-6380.conf" cluster-node-timeout 15000 maxclients 80000 # Generated by CONFIG REWRITE masterauth "pass" requirepass "pass"
先启动redis,
redis-server 6379/6379.conf
redis-server 6380/6380.conf
...
redis-server 6384/6384.conf
会看到进程
[root@localhost 6379]# ps -ef|grep redis root 3757 27288 0 17:00 pts/1 00:00:00 grep redis root 10469 1 0 16:32 ? 00:00:02 redis-server 0.0.0.0:6379 [cluster] root 10588 1 0 16:32 ? 00:00:02 redis-server 0.0.0.0:6380 [cluster] root 10657 1 0 16:32 ? 00:00:02 redis-server 0.0.0.0:6381 [cluster] root 10730 1 0 16:32 ? 00:00:02 redis-server 0.0.0.0:6382 [cluster] root 10812 1 0 16:32 ? 00:00:02 redis-server 0.0.0.0:6383 [cluster] root 10894 1 0 16:32 ? 00:00:02 redis-server 0.0.0.0:6384 [cluster]
执行redis的创建集群命令
cd /usr/local/redis3.2.2/src
./redis-trib.rb create --replicas 1 192.168.1.53:6379 192.168.1.53:6380 192.168.1.53:6381 192.168.1.53:6382 192.168.1.53:6383 192.168.1.53:6384
执行上面的命令的时候会报错,因为是执行的ruby的脚本,需要ruby的环境
错误内容:/usr/bin/env: ruby: No such file or directory
所以需要安装ruby的环境,这里推荐使用yum install ruby安装,ruby 版本装的是
ruby 2.3.4,用ruby -v来查看版本,如果版本低不行
然后再执行第6步的创建集群命令,还会报错,提示缺少rubygems组件,使用yum安装
错误内容:
./redis-trib.rb:24:in `require': no such file to load -- rubygems (LoadError)
from ./redis-trib.rb:24
yum install rubygems |
再次执行建集群命令,还会报错,提示不能加载redis,是因为缺少redis和ruby的接口,使用gem 安装
错误内容:
/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- redis (LoadError)
from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from ./redis-trib.rb:25
gem install redis --version 3.2.2
发现安装不了,网络不通,需要修改一下gem的源
gem sources --remove https://rubygems.org/
gem sources --add https://gems.ruby-china.org/
gem sources -l
再次输入建立集群命令,发现
[root@localhost src]# ./redis-trib.rb create --replicas 1 192.168.1.53:6379 192.168.1.53:6380 192.168.1.53:6381 192.168.1.53:6382 192.168.1.53:6383 192.168.1.53:6384 >>> Creating cluster [ERR] Sorry, can't connect to node 192.168.1.53:6379
这里有个大坑,集群没有启动前不能设置密码,于是先注掉密码,然后就可以建立了。
注意创建集群的语句只需要执行一次就行,下次redis重启就不用再执行。
集群检测:
[root@localhost 6379]# redis-trib.rb check 192.168.1.53:6379 >>> Performing Cluster Check (using node 192.168.1.53:6379) M: 2205ecf169d9f4b179c119e8ba378b9b19dd3bb7 192.168.1.53:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) S: e6ccd52511be1bd1bf27e6afd7a339a6febcedca 192.168.1.53:6383 slots: (0 slots) slave replicates 798efcd2daf6465e3014b3800d7c672f4a8a9a76 S: 00f5668d1a0e995c943fb6b329da68f3ef046a76 192.168.1.53:6382 slots: (0 slots) slave replicates 2205ecf169d9f4b179c119e8ba378b9b19dd3bb7 M: 798efcd2daf6465e3014b3800d7c672f4a8a9a76 192.168.1.53:6380 slots:5461-10922 (5462 slots) master 1 additional replica(s) S: e084ed6aaad61ba5af4ef0f7a652582f682c5fb9 192.168.1.53:6384 slots: (0 slots) slave replicates 92da6f66df3c4955ac3a14a28a1ca7225560f960 M: 92da6f66df3c4955ac3a14a28a1ca7225560f960 192.168.1.53:6381 slots:10923-16383 (5461 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
[root@localhost 6379]# redis-trib.rb info 192.168.1.53:6379 192.168.1.53:6379 (2205ecf1...) -> 5 keys | 5461 slots | 1 slaves. 192.168.1.53:6380 (798efcd2...) -> 9 keys | 5462 slots | 1 slaves. 192.168.1.53:6381 (92da6f66...) -> 5 keys | 5461 slots | 1 slaves. [OK] 19 keys in 3 masters. 0.00 keys per slot on average.
最后把密码设置上去:
需要连接到每台redis后执行:
config set masterauth pass
config set requirepass pass
然后统一保存到配置文件上
config rewrite
以集群方式登录
-c是以集群方式登录
例如:客户端登录6379端口的,设置的数据应该存放在7001上则会报错请转到7001。而加上-c启动则会自动切换到7001客户端保存。
redis-cli -c -p 6379 -a com.123 --raw
[root@localhost 6379]# redis-cli -c -p 6379 127.0.0.1:6379> set hello world OK 127.0.0.1:6379> set hello1 world -> Redirected to slot [11613] located at 192.168.1.53:6381 OK 192.168.1.53:6381> get hello -> Redirected to slot [866] located at 192.168.1.53:6379 "world" 192.168.1.53:6379> get hello1 -> Redirected to slot [11613] located at 192.168.1.53:6381 "world"
说明集群运作正常。