微服务轮子项目(29) -Redis 单机、主从复制、哨兵、cluster集群、持久化方案(上)

本文涉及的产品
云数据库 Redis 版,社区版 2GB
推荐场景:
搭建游戏排行榜
简介: 微服务轮子项目(29) -Redis 单机、主从复制、哨兵、cluster集群、持久化方案

1. 单机

单机没密码纯内存配置:

bind 0.0.0.0
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize no
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile "6379.log"
databases 16
save ""
always-show-logo yes
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dir ./
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
slave-lazy-flush no
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble no
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync ye

2. 主从复制

举例:

  • 1主1从、带密码、RDB-AOF混合持久化
  • 主节点端口为6379,从节点端口为6380

2.1 主节点配置

vim /o
bind 0.0.0.0
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize no
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile "6379.log"
dbfilename dump-6379.rdb
requirepass 1q2w3e4r
masterauth 1q2w3e4r
databases 16
always-show-logo yes
save ""
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dir ./
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
slave-lazy-flush no
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes

2.2 主节点启动文件

vim start-6379.sh
PORT=6379
docker stop redis-${PORT}
docker rm redis-${PORT}
docker run --name redis-${PORT} \
           -p ${PORT}:${PORT} \
           -v /opt/redis/conf/${PORT}.conf:/etc/redis/redis.conf \
           -v /opt/redis/data:/data \
           -d redis:4.0 \
           redis-server /etc/redis/redis.conf

2.3 从节点配置

vim /opt/redis/conf/6380.conf
bind 0.0.0.0
protected-mode yes
port 6380
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize no
supervised no
pidfile /var/run/redis_6380.pid
loglevel notice
logfile "6380.log"
dbfilename dump-6380.rdb
slaveof 192.168.28.130 6379
requirepass 1q2w3e4r
masterauth 1q2w3e4r
databases 16
always-show-logo yes
save ""
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dir ./
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
slave-lazy-flush no
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes

slaveof:启用主从模式配置主库ip和端口

slave-read-only:从节点是否只读

masterauth:设置访问master服务器的密码,如果主节点设置了密码必需添加该参数才能同步数据

2.4 从节点启动文件

vim start-6380.sh
PORT=6380
docker stop redis-${PORT}
docker rm redis-${PORT}
docker run --name redis-${PORT} \
           -p ${PORT}:${PORT} \
           -v /opt/redis/conf/${PORT}.conf:/etc/redis/redis.conf \
           -v /opt/redis/data:/data \
           -d redis:4.0 \
           redis-server /etc/redis/redis.conf

2.5 查看主从节点状态

##进入主节点容器
docker exec -it redis-6379 bash
##通过密码登录redis
redis-cli -a 1q2w3e4r
##查看主从信息
info replication

3. 主从复制+哨兵

sentinel是为了解决主从模式下,如果主节点由于故障下线了,自动升级从节点为主节点。

每个sentinel节点其实就是一个redis实例,与主从节点不同的是sentinel节点作用是用于监控redis数据节点的,而sentinel节点集合则表示监控一组主从redis实例多个sentinel监控节点的集合,比如有主节点master和从节点slave-1slave-2

sentinel主节点端口为26379,sentinel从节点端口为26380

3.1 主节点配置

vim /opt/redis/conf/sentinel-26379.conf
port 26379
logfile "26379.log"
sentinel myid 5fe8c1456f5080df3e485d44447e812e97ecd4d1
sentinel deny-scripts-reconfig yes
daemonize no
pidfile "/var/run/redis-sentinel.pid"
dir "/tmp"
sentinel monitor mymaster 192.168.28.130 6379 1
sentinel down-after-milliseconds mymaster 5000
sentinel auth-pass mymaster 1q2w3e4r

myid:唯一id,sentinel集群的各自myid参数必需唯一

sentinel monitor:只需配置主节点即刻,端口后面的数字代表指明当有多少个sentinel认为一个master失效时,master才算真正失效

sentinel down-after-milliseconds:需要多少失效时间,一个master才会被这个sentinel主观地认为是不可用的(默认30秒)

sentinel auth-pass:设置连接master和slave时的密码

3.2 主节点启动文件

vim start-sentinel-6379.sh
PORT=26379
docker stop redis-sentinel-${PORT}
docker rm redis-sentinel-${PORT}
docker run --name redis-sentinel-${PORT} \
           -p ${PORT}:${PORT} \
           -v /opt/redis/conf/sentinel-${PORT}.conf:/etc/redis/sentinel.conf \
           -v /opt/redis/data:/data \
           -d redis:4.0 \
           redis-sentinel /etc/redis/sentinel.conf

3.3 从节点配置

vim /opt/redis/conf/sentinel-26380.conf
port 26380
logfile "26380.log"
sentinel myid 5fe8c1456f5080df3e485d44447e812e97ecd4d2
sentinel deny-scripts-reconfig yes
daemonize no
pidfile "/var/run/redis-sentinel.pid"
dir "/tmp"
sentinel monitor mymaster 192.168.28.130 6379 1
sentinel down-after-milliseconds mymaster 5000
sentinel auth-pass mymaster 1q2w3e4r

3.4 从节点启动文件

vim start-sentinel-6380.sh
PORT=26380
docker stop redis-sentinel-${PORT}
docker rm redis-sentinel-${PORT}
docker run --name redis-sentinel-${PORT} \
           -p ${PORT}:${PORT} \
           -v /opt/redis/conf/sentinel-${PORT}.conf:/etc/redis/sentinel.conf \
           -v /opt/redis/data:/data \
           -d redis:4.0 \
           redis-sentinel /etc/redis/sentinel.conf

3.5 查看哨兵状态

## 进入主节点容器
docker exec -it redis-sentinel-26379 bash
## 登录redis
redis-cli -p 26379
## 查看sentinel信息
info sentinel

3.6 spring boot 工程集成哨兵集群

把单机的spring.redis.hostspring.redis.port配置改成以下

##### redis配置
spring.redis.sentinel.master=mymaster
spring.redis.sentinel.nodes=192.168.28.130:26379,192.168.28.130:26380
spring.redis.password=1q2w3e4r

4. cluster集群

以下例子用的是伪集群,真正的集群放到不同的机器即可。

  • 端口是7001-7006
  • 工作目录:/opt/redis-cluster

4.1 创建文件夹

首先创建一堆对应端口的文件夹,编辑创建脚本vim create.sh内容如下:

for i in `seq 7001 7006`
do
 mkdir -p ${i}/data
done

执行脚本:

sh create.sh

4.2 创建docker-compose配置

编辑环境配置vim .env内容如下

redis_path=/opt/redis-cluster

编辑vim docker-compose.yml内容如下:

version: '3.4'
x-image:
 &default-image
 publicisworldwide/redis-cluster
x-restart:
 &default-restart
 always
x-netmode:
 &default-netmode
 host
services:
 redis1:
  image: *default-image
  network_mode: *default-netmode
  restart: *default-restart
  volumes:
  - ${redis_path}/7001/data:/data
  environment:
  - REDIS_PORT=7001
 redis2:
  image: *default-image
  network_mode: *default-netmode
  restart: *default-restart
  volumes:
  - ${redis_path}/7002/data:/data
  environment:
  - REDIS_PORT=7002
 redis3:
  image: *default-image
  network_mode: *default-netmode
  restart: *default-restart
  volumes:
  - ${redis_path}/7003/data:/data
  environment:
  - REDIS_PORT=7003
 redis4:
  image: *default-image
  network_mode: *default-netmode
  restart: *default-restart
  volumes:
  - ${redis_path}/7004/data:/data
  environment:
  - REDIS_PORT=7004
 redis5:
  image: *default-image
  network_mode: *default-netmode
  restart: *default-restart
  volumes:
  - ${redis_path}/7005/data:/data
  environment:
  - REDIS_PORT=7005
 redis6:
  image: *default-image
  network_mode: *default-netmode
  restart: *default-restart
  volumes:
  - ${redis_path}/7006/data:/data
  environment:
  - REDIS_PORT=7006

4.3 启动所有redis

执行以下命令

docker-compose up -d

4.4 部署cluster

通过inem0o/redis-trib镜像,编辑脚本vim redis-trib.sh内容如下:

docker run --rm -it inem0o/redis-trib create --replicas 1 192.168.28.130:7001 192.168.28.130:7002 192.168.28.130:7003 192.168.28.130:7004 192.168.28.130:7005 192.168.28.130:7006

运行脚本sh redis-trib.sh:

输入yes:


相关实践学习
基于Redis实现在线游戏积分排行榜
本场景将介绍如何基于Redis数据库实现在线游戏中的游戏玩家积分排行榜功能。
云数据库 Redis 版使用教程
云数据库Redis版是兼容Redis协议标准的、提供持久化的内存数据库服务,基于高可靠双机热备架构及可无缝扩展的集群架构,满足高读写性能场景及容量需弹性变配的业务需求。 产品详情:https://www.aliyun.com/product/kvstore     ------------------------------------------------------------------------- 阿里云数据库体验:数据库上云实战 开发者云会免费提供一台带自建MySQL的源数据库 ECS 实例和一台目标数据库 RDS实例。跟着指引,您可以一步步实现将ECS自建数据库迁移到目标数据库RDS。 点击下方链接,领取免费ECS&RDS资源,30分钟完成数据库上云实战!https://developer.aliyun.com/adc/scenario/51eefbd1894e42f6bb9acacadd3f9121?spm=a2c6h.13788135.J_3257954370.9.4ba85f24utseFl
目录
相关文章
|
3天前
|
负载均衡 监控 NoSQL
Redis的几种主要集群方案
【5月更文挑战第15天】Redis集群方案包括主从复制(基础,读写分离,手动故障恢复)、哨兵模式(自动高可用,自动故障转移)和Redis Cluster(官方分布式解决方案,自动分片、容错和扩展)。此外,还有Codis、Redisson和Twemproxy等工具用于代理分片和负载均衡。选择方案需考虑应用场景、数据量和并发需求,权衡可用性、性能和扩展性。
33 2
|
3天前
|
存储 缓存 NoSQL
【技术分享】求取列表需求的redis缓存方案
【技术分享】求取列表需求的redis缓存方案
14 0
|
3天前
|
存储 NoSQL Redis
Redis源码、面试指南(5)多机数据库、复制、哨兵、集群(下)
Redis源码、面试指南(5)多机数据库、复制、哨兵、集群
19 1
|
3天前
|
监控 NoSQL Redis
Redis源码、面试指南(5)多机数据库、复制、哨兵、集群(上)
Redis源码、面试指南(5)多机数据库、复制、哨兵、集群
34 0
|
3天前
|
存储 监控 NoSQL
Redis哨兵&分片集群
Redis哨兵&分片集群
20 0
|
3天前
|
负载均衡 NoSQL 关系型数据库
深入浅出Redis(六):Redis的主从架构与主从复制原理
深入浅出Redis(六):Redis的主从架构与主从复制原理
|
3天前
|
存储 缓存 NoSQL
Redis实现延迟任务的几种方案
Redis实现延迟任务的几种方案
|
3天前
|
监控 NoSQL 算法
Redis 搭建哨兵集群
Redis 搭建哨兵集群
24 1
|
3天前
|
存储 缓存 运维
软件体系结构 - 缓存技术(5)Redis Cluster
【4月更文挑战第20天】软件体系结构 - 缓存技术(5)Redis Cluster
142 10
|
3天前
|
Java Maven 微服务
微服务项目-将普通文件夹设为模块与添加services窗口
微服务项目-将普通文件夹设为模块与添加services窗口
10 0