Sentinel是什么？Redis高可用是什么？
本篇博文分享的是一主二从三哨兵模式。至于为什么用三个哨兵，同第一段。本文是模拟环境，都是一个服务器上面。
【1】POM文件和配置
① pom文件
<parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.1.1.RELEASE</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>com.boot.redis</groupId> <artifactId>boot-redis</artifactId> <version>0.0.1-SNAPSHOT</version> <name>boot-redis</name> <description>Demo project for Spring Boot</description> <properties> <java.version>1.8</java.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-redis</artifactId> <exclusions> <exclusion> <groupId>redis.clients</groupId> <artifactId>jedis</artifactId> </exclusion> <exclusion> <groupId>io.lettuce</groupId> <artifactId>lettuce-core</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>redis.clients</groupId> <artifactId>jedis</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-pool2</artifactId> <version>2.5.0</version> <!--<version>2.4.2</version>--> </dependency> </dependencies>
② yml简单配置
spring: redis: host: 192.168.2.110 #哨兵模式下不用配置 port: 6379 # 哨兵模式下不用配置 password: admin jedis: pool: #最大连接数 max-active: 1024 #最大阻塞等待时间(负数表示没限制) max-wait: 20000 #最大空闲 max-idle: 200 #最小空闲 min-idle: 10 sentinel: master: mymaster nodes: 192.168.2.110:26379,192.168.2.110:26380,192.168.2.110:26381 server: port: 8088
至于客户端使用jedis或者lettuce等相关详细配置，这里回头更新。本文重点不在这里。
③ redis服务配置
6379.conf(默认为master)：
port 6379 masterauth admin requirepass admin protected-mode yes # 服务器ip地址 bind 192.168.2.110
6380配置(slave):
port 6380 masterauth admin requirepass admin protected-mode yes # 服务器ip地址 bind 192.168.2.110 slaveof 192.168.2.110 6379
6381同6380，只是端口不一致。若找不到在redis.conf中对应位置，博文末尾GitHub中已经提交相关配置。
④ 哨兵配置
sentinel.conf(默认端口26379)：
port 26379 sentinel monitor mymaster 192.168.2.110 6379 2 sentinel auth-pass mymaster admin sentinel down-after-milliseconds mymaster 15000 sentinel parallel-syncs mymaster 1 sentinel failover-timeout mymaster 80000 bind 192.168.2.110 protected-mode yes
其他两个哨兵配置文件类似，只是需要修改一下端口分别为26380和26381。
⑤ RedisConfig类
@Configuration @EnableAutoConfiguration public class RedisConfig { private static Logger logger = LoggerFactory.getLogger(RedisConfig.class); @Value("#{'${spring.redis.sentinel.nodes}'.split(',')}") private List<String> nodes; @Bean @ConfigurationProperties(prefix="spring.redis") public JedisPoolConfig getRedisConfig(){ JedisPoolConfig config = new JedisPoolConfig(); return config; } @Bean public RedisSentinelConfiguration sentinelConfiguration(){ RedisSentinelConfiguration redisSentinelConfiguration = new RedisSentinelConfiguration(); //配置matser的名称 redisSentinelConfiguration.master("mymaster"); //配置redis的哨兵sentinel Set<RedisNode> redisNodeSet = new HashSet<>(); nodes.forEach(x->{ redisNodeSet.add(new RedisNode(x.split(":")[0],Integer.parseInt(x.split(":")[1]))); }); logger.info("redisNodeSet -->"+redisNodeSet); redisSentinelConfiguration.setSentinels(redisNodeSet); return redisSentinelConfiguration; } @Bean public JedisConnectionFactory jedisConnectionFactory(JedisPoolConfig jedisPoolConfig,RedisSentinelConfiguration sentinelConfig) { JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory(sentinelConfig,jedisPoolConfig); return jedisConnectionFactory; } }
然后在服务器依次启动redis服务6379、6380、6381和哨兵服务26379/26380以及26381。
此时可以通过命令窗口查看主从以及哨兵状况，可以使用命令进行主从切换，查看哨兵选举。
另外将项目打包放到服务器运行，查看当master挂掉时，SpringBoot所获取到的信息与转变。
【2】几点说明
① 哨兵之间通信
哨兵启动后，相互之间是保持通信的：
如下所示，一个哨兵挂掉重启后将会从其他哨兵获取最新信息：
② 当master挂掉之后，三个哨兵需要选举出来一个leader进行主从转移
③ 如果哨兵挂掉，则MasterListener将会隔5秒重试
④ 如果master挂掉，MasterListener将会通过Sentinel获取最新的master信息并重置JedisSentinelPool
所以不用担心master挂掉，SpringBoot会怎么办。有MasterListener在，一切OK。
⑤ 哨兵配置动态修改
哨兵启动后，配置文件是会被动态改变的，如下所示：
port 26379 sentinel myid 8ac857af0b63edc73e07cce9c11462eef047704d sentinel monitor mymaster 127.0.0.1 6380 2 sentinel down-after-milliseconds mymaster 15000 sentinel failover-timeout mymaster 80000 sentinel auth-pass mymaster admin bind 192.168.2.110 protected-mode yes # Generated by CONFIG REWRITE dir "/root" sentinel config-epoch mymaster 1 sentinel leader-epoch mymaster 1 sentinel known-slave mymaster 192.168.2.110 6379 sentinel known-slave mymaster 127.0.0.1 6381 sentinel known-sentinel mymaster 192.168.2.110 26381 b3d9ee9efae47556349a733ed668385d8083b5cc sentinel known-sentinel mymaster 192.168.2.110 26380 e7a100d97d70834ea3b117cf23a4b08f7a46b425 sentinel current-epoch 1
【3】几个类
这里面有几个类十分重要 ，建议查看源码跟踪一番。
① JedisConnectionFactory
时序图如下：
重点在步骤12中，MasterListener继承了Thread，它是个单独的线程，通过Jedis监听Redis的哨兵，如果收到master改变的消息，那么会修改JedisFactory。
如下图所示是MasterListener收到master节点变化的消息：
想要debug跟踪源码的同学可以在下图处打上断点：
② JedisSentinelPool
其构造方法如下：
public JedisSentinelPool(String masterName, Set<String> sentinels, final GenericObjectPoolConfig poolConfig, final int connectionTimeout, final int soTimeout, final String password, final int database, final String clientName) { this.poolConfig = poolConfig; this.connectionTimeout = connectionTimeout; this.soTimeout = soTimeout; this.password = password; this.database = database; this.clientName = clientName; HostAndPort master = initSentinels(sentinels, masterName); initPool(master); }
其中initSentinels(sentinels, masterName)就是根据哨兵和master名字来获取master的信息–host和port。然后根据获取的maser来初始化pool(所以yml中不用配置spring.redis.host和spring.redis.port)。
从应用启动也可以看到这一点：
获取HostAndPort master方法如下所示：
private HostAndPort initSentinels(Set<String> sentinels, final String masterName) { HostAndPort master = null; boolean sentinelAvailable = false; log.info("Trying to find master from available Sentinels..."); //遍历sentinels，获取master，然后基于该master创建pool for (String sentinel : sentinels) { final HostAndPort hap = HostAndPort.parseString(sentinel); log.fine("Connecting to Sentinel " + hap); Jedis jedis = null; try { jedis = new Jedis(hap.getHost(), hap.getPort()); //根据哨兵的host port 以及masterName获取masterAddr List<String> masterAddr = jedis.sentinelGetMasterAddrByName(masterName); // connected to sentinel... sentinelAvailable = true; if (masterAddr == null || masterAddr.size() != 2) { log.warning("Can not get master addr, master name: " + masterName + ". Sentinel: " + hap + "."); continue; } //转换格式 master = toHostAndPort(masterAddr); log.fine("Found Redis master at " + master); break; } catch (JedisException e) { // resolves #1036, it should handle JedisException there's another chance // of raising JedisDataException log.warning("Cannot get master address from sentinel running @ " + hap + ". Reason: " + e + ". Trying next one."); } finally { if (jedis != null) { jedis.close(); } } } if (master == null) { if (sentinelAvailable) { // can connect to sentinel, but master name seems to not // monitored throw new JedisException("Can connect to sentinel, but " + masterName + " seems to be not monitored..."); } else { throw new JedisConnectionException("All sentinels down, cannot determine where is " + masterName + " master is running..."); } } log.info("Redis master running at " + master + ", starting Sentinel listeners..."); //如下，启动每个哨兵的监听 for (String sentinel : sentinels) { final HostAndPort hap = HostAndPort.parseString(sentinel); MasterListener masterListener = new MasterListener(masterName, hap.getHost(), hap.getPort()); // whether MasterListener threads are alive or not, process can be stopped masterListener.setDaemon(true); masterListeners.add(masterListener); masterListener.start(); } return master; }
初始化pool的方法如下所示：
private void initPool(HostAndPort master) { if (!master.equals(currentHostMaster)) { currentHostMaster = master; //判断JedisFactory是否为null，为null则使用master创建 if (factory == null) { factory = new JedisFactory(master.getHost(), master.getPort(), connectionTimeout, soTimeout, password, database, clientName, false, null, null, null); initPool(poolConfig, factory); } else { factory.setHostAndPort(currentHostMaster); // although we clear the pool, we still have to check the // returned object // in getResource, this call only clears idle instances, not // borrowed instances internalPool.clear(); } log.info("Created JedisPool to master at " + master); } }
③ MasterListener
MasterListener 是一个继承自Thread 的线程类，用来监听哨兵的master改变信息。 收到master改变的消息后，调用initPool方法修改JedisFactory，进而会影响connection的获取。
protected class MasterListener extends Thread { protected String masterName; protected String host; protected int port; protected long subscribeRetryWaitTimeMillis = 5000; protected volatile Jedis j; protected AtomicBoolean running = new AtomicBoolean(false); protected MasterListener() { } public MasterListener(String masterName, String host, int port) { super(String.format("MasterListener-%s-[%s:%d]", masterName, host, port)); this.masterName = masterName; this.host = host; this.port = port; } public MasterListener(String masterName, String host, int port, long subscribeRetryWaitTimeMillis) { this(masterName, host, port); this.subscribeRetryWaitTimeMillis = subscribeRetryWaitTimeMillis; } //主要功能如下 @Override public void run() { running.set(true); while (running.get()) { //拿到一个Jedis j = new Jedis(host, port); try { // double check that it is not being shutdown if (!running.get()) { break; } //发布订阅模式 j.subscribe(new JedisPubSub() { @Override public void onMessage(String channel, String message) { log.fine("Sentinel " + host + ":" + port + " published: " + message + "."); String[] switchMasterMsg = message.split(" "); if (switchMasterMsg.length > 3) { if (masterName.equals(switchMasterMsg[0])) { //这里，重置initPool initPool(toHostAndPort(Arrays.asList(switchMasterMsg[3], switchMasterMsg[4]))); } else { log.fine("Ignoring message on +switch-master for master name " + switchMasterMsg[0] + ", our master name is " + masterName); } } else { log.severe("Invalid message received on Sentinel " + host + ":" + port + " on channel +switch-master: " + message); } } }, "+switch-master"); } catch (JedisConnectionException e) { if (running.get()) { log.log(Level.SEVERE, "Lost connection to Sentinel at " + host + ":" + port + ". Sleeping 5000ms and retrying.", e); try { Thread.sleep(subscribeRetryWaitTimeMillis); } catch (InterruptedException e1) { log.log(Level.SEVERE, "Sleep interrupted: ", e1); } } else { log.fine("Unsubscribing from Sentinel at " + host + ":" + port); } } finally { j.close(); } } } public void shutdown() { try { log.fine("Shutting down listener on " + host + ":" + port); running.set(false); // This isn't good, the Jedis object is not thread safe if (j != null) { j.disconnect(); } } catch (Exception e) { log.log(Level.SEVERE, "Caught exception while shutting down: ", e); } } }
