【Mongodb】 Replica set的自动故障切换

本文涉及的产品
云数据库 MongoDB,独享型 2核8GB
推荐场景:
构建全方位客户视图
日志服务 SLS,月写入数据量 50GB 1个月
简介:

Replica set 为我们提供了自动故障切换功能,这个机制是由mongodb自己来操作的,它根据从库的优先级或者数据新鲜度(也就是最新的从主库同步数据的那个节点)来选择primary,而当以前的primary起来之后,会成为secondary ,接受新的primary 的日志。

22664653_201110312221291.jpg

                                                              完整的replica sets

22664653_201110312221371.jpg

                                                             primary 当机

22664653_201110312221431.jpg

                   mongodb 会根据数据的新鲜度来选择下一个主库


接上一篇文章,搭建好了replica set,查看端口为 27018 27020两个服务的状态:

[mongodb@rac4 bin]$ ./mongo 127.0.0.1:27018

MongoDB shell version: 2.0.1

connecting to: 127.0.0.1:27018/test

PRIMARY> db.isMaster();

{

        "setName" : "myset",

        "ismaster" : true,  --为主库

        "secondary" : false,

        "hosts" : [

                "10.250.7.220:27018",

                "10.250.7.220:27020",

                "10.250.7.220:27019"

        ],

        "primary" : "10.250.7.220:27018",

        "me" : "10.250.7.220:27018",

        "maxBsonObjectSize" : 16777216,

        "ok" : 1

}

PRIMARY> exit

bye

[mongodb@rac4 bin]$ ./mongo 127.0.0.1:27020

MongoDB shell version: 2.0.1

connecting to: 127.0.0.1:27020/test

SECONDARY> 

SECONDARY> db.isMaster();

{

        "setName" : "myset",

        "ismaster" : false,

        "secondary" : true, --为从库

        "hosts" : [

                "10.250.7.220:27020",

                "10.250.7.220:27019",

                "10.250.7.220:27018"

        ],

        "primary" : "10.250.7.220:27018",

        "me" : "10.250.7.220:27020",

        "maxBsonObjectSize" : 16777216,

        "ok" : 1

}


PRIMARY> 手工杀掉primary 

[root@rac4 ~]# ps -ef | grep 27018 

mongodb  14826 14794  1 20:24 pts/4    00:00:05 ./mongod --dbpath /opt/mongodata/r1 --port 27018 --replSet myset --rest

mongodb  14999 14430  0 20:28 pts/2    00:00:00 ./mongo 127.0.0.1:27018

[root@rac4 ~]# kill -9 14826 14794

[root@rac4 ~]# ps -ef | grep mongodb |grep -v root

mongodb  14883 14853  1 20:26 pts/7    00:00:05 ./mongod --dbpath /opt/mongodata/r2 --port 27019 --replSet myset --rest

mongodb  14901 14548  1 20:27 pts/6    00:00:07 ./mongod --dbpath /opt/mongodata/r3 --port 27020 --replSet myset --rest

mongodb  14999 14430  0 20:28 pts/2    00:00:00 ./mongo 127.0.0.1:27018

mongodb  15102 15072  0 20:30 pts/5    00:00:00 ./mongo 127.0.0.1:27019

mongodb  15136 15106  0 20:30 pts/8    00:00:00 ./mongo 127.0.0.1:27020

[root@rac4 ~]# 

27019 端口的mongodb 输出日志显示的选择10.250.7.220 作为主库的日志记录

Mon Oct 31 20:27:59 [FileAllocator] allocating new datafile /opt/mongodata/r2/local.2, filling with zeroes...

Mon Oct 31 20:27:59 [rsHealthPoll] replSet info member 10.250.7.220:27018 is up

Mon Oct 31 20:27:59 [rsHealthPoll] replSet member 10.250.7.220:27018 is now in state SECONDARY

Mon Oct 31 20:27:59 [rsHealthPoll] replSet info 10.250.7.220:27020 is down (or slow to respond): still initializing

Mon Oct 31 20:27:59 [rsHealthPoll] replSet member 10.250.7.220:27020 is now in state DOWN

Mon Oct 31 20:28:01 [initandlisten] connection accepted from 10.250.7.220:10857 #3

Mon Oct 31 20:28:05 [conn2] replSet RECOVERING

Mon Oct 31 20:28:05 [conn2] replSet info voting yea for 10.250.7.220:27018 (0)

Mon Oct 31 20:28:07 [rsHealthPoll] replSet member 10.250.7.220:27018 is now in state PRIMARY

Mon Oct 31 20:28:09 [FileAllocator] done allocating datafile /opt/mongodata/r2/local.2, size: 1024MB,  took 10.89 secs

Mon Oct 31 20:28:10 [rsSync] ******

Mon Oct 31 20:28:10 [rsSync] replSet initial sync pending

Mon Oct 31 20:28:10 [rsSync] replSet syncing to: 10.250.7.220:27018

Mon Oct 31 20:28:10 [rsSync] build index local.me { _id: 1 }

Mon Oct 31 20:28:10 [rsSync] build index done 0 records 0.001 secs

Mon Oct 31 20:28:10 [rsSync] replSet initial sync drop all databases

Mon Oct 31 20:28:10 [rsSync] dropAllDatabasesExceptLocal 1

Mon Oct 31 20:28:10 [rsSync] replSet initial sync clone all databases

Mon Oct 31 20:28:10 [rsSync] replSet initial sync query minValid

Mon Oct 31 20:28:10 [rsSync] replSet initial oplog application from 10.250.7.220:27018 starting at Oct 31 20:27:53:1 to Oct 31 20:27:53:1

Mon Oct 31 20:28:13 [rsHealthPoll] replSet info member 10.250.7.220:27020 is up

Mon Oct 31 20:28:13 [rsHealthPoll] replSet member 10.250.7.220:27020 is now in state STARTUP2

Mon Oct 31 20:28:14 [rsSync] replSet initial sync finishing up

Mon Oct 31 20:28:14 [rsSync] replSet set minValid=4eae9449:1

Mon Oct 31 20:28:14 [rsSync] build index local.replset.minvalid { _id: 1 }

Mon Oct 31 20:28:14 [rsSync] build index done 0 records 0.005 secs

Mon Oct 31 20:28:14 [rsSync] replSet initial sync done

Mon Oct 31 20:28:15 [rsSync] replSet syncing to: 10.250.7.220:27018

Mon Oct 31 20:28:15 [rsSync] replSet SECONDARY

Mon Oct 31 20:28:15 [rsHealthPoll] replSet member 10.250.7.220:27020 is now in state RECOVERING

Mon Oct 31 20:28:26 [clientcursormon] mem (MB) res:16 virt:2677 mapped:1232

Mon Oct 31 20:28:52 [initandlisten] connection accepted from 10.250.7.220:10872 #4

Mon Oct 31 20:28:52 [initandlisten] connection accepted from 10.250.7.220:10873 #5

Mon Oct 31 20:28:52 [rsGhostSync] handshake between 2 and 10.250.7.220:27018

Mon Oct 31 20:28:53 [slaveTracking] build index local.slaves { _id: 1 }

Mon Oct 31 20:28:53 [slaveTracking] build index done 0 records 0.003 secs

Mon Oct 31 20:28:55 [conn5] end connection 10.250.7.220:10873

Mon Oct 31 20:28:55 [conn4] end connection 10.250.7.220:10872

Mon Oct 31 20:28:57 [rsHealthPoll] replSet member 10.250.7.220:27020 is now in state SECONDARY

Mon Oct 31 20:29:27 [clientcursormon] mem (MB) res:19 virt:2693 mapped:1232

Mon Oct 31 20:30:21 [initandlisten] connection accepted from 127.0.0.1:44672 #6

Mon Oct 31 20:33:35 [conn2] end connection 10.250.7.220:42493

Mon Oct 31 20:33:35 [rsSync] replSet syncThread: 10278 dbclient error communicating with server: 10.250.7.220:27018

Mon Oct 31 20:33:35 [rsHealthPoll] DBClientCursor::init call() failed

Mon Oct 31 20:33:35 [rsHealthPoll] replSet info 10.250.7.220:27018 is down (or slow to respond): DBClientBase::findN: transport error: 10.250.7.220:27018 query: { replSetHeartbeat: "myset", v: 1, pv: 1, checkEmpty: false, from: "10.250.7.220:27019" }

Mon Oct 31 20:33:35 [rsHealthPoll] replSet member 10.250.7.220:27018 is now in state DOWN

Mon Oct 31 20:33:35 [rsMgr] not electing self, 10.250.7.220:27020 would veto

Mon Oct 31 20:33:36 [conn3] replSet info voting yea for 10.250.7.220:27020 (2)

Mon Oct 31 20:33:37 [rsHealthPoll] replSet member 10.250.7.220:27020 is now in state PRIMARY

Mon Oct 31 20:33:46 [rsSync] replSet syncing to: 10.250.7.220:27020

Mon Oct 31 20:34:27 [clientcursormon] mem (MB) res:19 virt:2693 mapped:1232


27020 端口的mongodb 输出日志显示的选择10.250.7.220 作为主库的日志记录

Mon Oct 31 20:33:35 [rsSync] replSet syncThread: 10278 dbclient error communicating with server: 10.250.7.220:27018

Mon Oct 31 20:33:36 [rsHealthPoll] DBClientCursor::init call() failed

Mon Oct 31 20:33:36 [rsHealthPoll] replSet info 10.250.7.220:27018 is down (or slow to respond): DBClientBase::findN: transport error: 10.250.7.220:27018 query: { replSetHeartbeat: "myset", v: 1, pv: 1, checkEmpty: false, from: "10.250.7.220:27020" }

Mon Oct 31 20:33:36 [rsHealthPoll] replSet member 10.250.7.220:27018 is now in state DOWN

Mon Oct 31 20:33:36 [rsMgr] replSet info electSelf 2

Mon Oct 31 20:33:36 [rsMgr] replSet PRIMARY

Mon Oct 31 20:33:46 [initandlisten] connection accepted from 10.250.7.220:37261 #5

Mon Oct 31 20:33:47 [slaveTracking] build index local.slaves { _id: 1 }

Mon Oct 31 20:33:47 [slaveTracking] build index done 0 records 0.001 secs

Mon Oct 31 20:33:48 [clientcursormon] mem (MB) res:19 virt:2692 mapped:1232

Mon Oct 31 20:34:35 [conn4] end connection 127.0.0.1:17500

Mon Oct 31 20:34:37 [initandlisten] connection accepted from 127.0.0.1:36525 #6

进入数据库查看:

[mongodb@rac4 bin]$ ./mongo 127.0.0.1:27020

MongoDB shell version: 2.0.1

connecting to: 127.0.0.1:27020/test

PRIMARY> 

PRIMARY> 

PRIMARY> db.isMaster();

{

        "setName" : "myset",

        "ismaster" : true,--成为主库master

        "secondary" : false,

        "hosts" : [

                "10.250.7.220:27020",

                "10.250.7.220:27019",

                "10.250.7.220:27018"

        ],

        "primary" : "10.250.7.220:27020",

        "me" : "10.250.7.220:27020",

        "maxBsonObjectSize" : 16777216,

        "ok" : 1

}

PRIMARY> 

重新启动端口为27018的mongodb的数据库服务:从日志中可以看出其进行恢复的操作记录

[mongodb@rac4 bin]$ ./mongod --dbpath /opt/mongodata/r1 --port 27018  --rest --replSet myset &

[1] 16290

[mongodb@rac4 bin]$ Mon Oct 31 20:48:32 [initandlisten] MongoDB starting : pid=16290 port=27018 dbpath=/opt/mongodata/r1 64-bit host=rac4

Mon Oct 31 20:48:32 [initandlisten] db version v2.0.1, pdfile version 4.5

Mon Oct 31 20:48:32 [initandlisten] git version: 3a5cf0e2134a830d38d2d1aae7e88cac31bdd684

Mon Oct 31 20:48:32 [initandlisten] build info: Linux bs-linux64.10gen.cc 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41

Mon Oct 31 20:48:32 [initandlisten] options: { dbpath: "/opt/mongodata/r1", port: 27018, replSet: "myset", rest: true }

Mon Oct 31 20:48:32 [initandlisten] journal dir=/opt/mongodata/r1/journal

Mon Oct 31 20:48:32 [initandlisten] recover begin

Mon Oct 31 20:48:32 [initandlisten] recover lsn: 231055

Mon Oct 31 20:48:32 [initandlisten] recover /opt/mongodata/r1/journal/j._0

Mon Oct 31 20:48:32 [initandlisten] recover skipping application of section seq:198962 < lsn:231055

Mon Oct 31 20:48:32 [initandlisten] recover cleaning up

Mon Oct 31 20:48:32 [initandlisten] removeJournalFiles

Mon Oct 31 20:48:32 [initandlisten] recover done

Mon Oct 31 20:48:32 [initandlisten] waiting for connections on port 27018

Mon Oct 31 20:48:32 [websvr] admin web console waiting for connections on port 28018

Mon Oct 31 20:48:32 [initandlisten] connection accepted from 127.0.0.1:11930 #1

Mon Oct 31 20:48:32 [rsStart] replSet STARTUP2

Mon Oct 31 20:48:32 [rsHealthPoll] replSet info member 10.250.7.220:27019 is up

Mon Oct 31 20:48:32 [rsHealthPoll] replSet member 10.250.7.220:27019 is now in state SECONDARY

Mon Oct 31 20:48:32 [rsHealthPoll] replSet info member 10.250.7.220:27020 is up

Mon Oct 31 20:48:32 [rsHealthPoll] replSet member 10.250.7.220:27020 is now in state PRIMARY

Mon Oct 31 20:48:32 [rsSync] replSet SECONDARY

Mon Oct 31 20:48:33 [initandlisten] connection accepted from 10.250.7.220:35971 #2

Mon Oct 31 20:48:34 [initandlisten] connection accepted from 10.250.7.220:35972 #3

Mon Oct 31 20:48:36 [rsSync] replSet syncing to: 10.250.7.220:27020

Mon Oct 31 20:48:36 [rsSync] build index local.me { _id: 1 }

Mon Oct 31 20:48:36 [rsSync] build index done 0 records 0 secs

[mongodb@rac4 bin]$ ./mongo 127.0.0.1:27018

MongoDB shell version: 2.0.1

connecting to: 127.0.0.1:27018/test

SECONDARY> 

SECONDARY> db.isMaster();

{

        "setName" : "myset",

        "ismaster" : false,   --端口为 27018的数据库服务变为从库

        "secondary" : true,

        "hosts" : [

                "10.250.7.220:27018",

                "10.250.7.220:27020",

                "10.250.7.220:27019"

        ],

        "primary" : "10.250.7.220:27020",

        "me" : "10.250.7.220:27018",

        "maxBsonObjectSize" : 16777216,

        "ok" : 1

}

SECONDARY> 

fj.png2.JPG

fj.png3.JPG



      本文转自yzy121403725 51CTO博客,原文链接:http://blog.51cto.com/lookingdream/1795019,如需转载请自行联系原作者





相关实践学习
MongoDB数据库入门
MongoDB数据库入门实验。
快速掌握 MongoDB 数据库
本课程主要讲解MongoDB数据库的基本知识,包括MongoDB数据库的安装、配置、服务的启动、数据的CRUD操作函数使用、MongoDB索引的使用(唯一索引、地理索引、过期索引、全文索引等)、MapReduce操作实现、用户管理、Java对MongoDB的操作支持(基于2.x驱动与3.x驱动的完全讲解)。 通过学习此课程,读者将具备MongoDB数据库的开发能力,并且能够使用MongoDB进行项目开发。 &nbsp; 相关的阿里云产品:云数据库 MongoDB版 云数据库MongoDB版支持ReplicaSet和Sharding两种部署架构,具备安全审计,时间点备份等多项企业能力。在互联网、物联网、游戏、金融等领域被广泛采用。 云数据库MongoDB版(ApsaraDB for MongoDB)完全兼容MongoDB协议,基于飞天分布式系统和高可靠存储引擎,提供多节点高可用架构、弹性扩容、容灾、备份回滚、性能优化等解决方案。 产品详情: https://www.aliyun.com/product/mongodb
相关文章
|
NoSQL MongoDB 数据库
MongoDB 自动删除集合中过期的数据——TTL索引
简介 ​ TTL (Time To Live, 有生命周期的) 索引是特殊单字段索引,MongoDB可以用来在一定时间后自动从集合中删除文档的特殊索引。 这对于某些类型的数据非常好,例如机器生成的事件数据,日志和会话信息,这些信息只需要在数据库中保留一段时间。 ​ 创建 TTL 索引,只需要在使用 db.collection.createIndex() 方法,对字段值为日期或者包含日期的数组设置 expireAfterSeconds 选项即可。 1、如果字段是一个数组,并有多个日期值时,MongoDB使用最低(即最早)日期值来计算失效阈值。 2、如果字段不是日期类型也不是一个包含日期的数组
974 0
|
NoSQL 算法 容灾
『MongoDB』MongoDB高可用部署架构——复制集篇(Replica Set)
读完这篇文章里你能收获到 1. MongoDB是如何通过复制集实现高可用的 2. 主节点宕机后如何通过选举做到故障恢复 3. 在复制集中常见的可调整参数有哪些 4. 在Linux原生环境搭建MongoDB复制集 5. 在Winodws环境搭建MongoDB复制集
992 1
『MongoDB』MongoDB高可用部署架构——复制集篇(Replica Set)
|
存储 NoSQL MongoDB
mongodb搭建Replica Set
mongodb搭建Replica Set 简单高效
233 0
|
SQL NoSQL JavaScript
MongoDB 自动增长
MongoDB 自动增长
117 0
|
存储 NoSQL 测试技术
MongoDB系列-解决面试中可能遇到的MongoDB复制集(replica set)问题
MongoDB复制集(replica set):MongoDB复制集维护相同数据集的一组mongod进程,复制集是生产部署的基础,具有数据冗余以及高可用性。
394 0
MongoDB系列-解决面试中可能遇到的MongoDB复制集(replica set)问题
|
运维 NoSQL MongoDB
(2)MongoDB副本集自动故障转移原理(含客户端)
前文我们搭建MongoDB三成员副本集,了解集群基本特性,今天我们围绕下图聊一聊背后的细节。
(2)MongoDB副本集自动故障转移原理(含客户端)
|
存储 NoSQL Shell
(1)解锁MongoDB replica set核心姿势
本文倒腾目前大热的MongoDB Replica Set集群,在倒腾的同时串讲一些 MongoDB特性。
(1)解锁MongoDB replica set核心姿势
|
存储 NoSQL 网络协议
MongoDB系列-复制集(Replica Set)应用部署(生产、测试、开发环境)
通过在不同的计算机上托管mongod实例来尽可能多地保持成员之间的分离。将虚拟机用于生产部署时,应将每个mongod实例放置在由冗余电源电路和冗余网络路径提供服务的单独主机服务器上,而且尽可能的将副本集的每个成员部署到自己的计算机绑定到标准的MongoDB端口27017。
555 0
|
NoSQL MongoDB
AutoScaling 通过lifecycleHook自动添加MongoDB白名单
本文将为您介绍如何通过lifecycleHook实现弹性伸缩组中实例自动加入/移除MongoDB白名单。 相关介绍 lifecycleHook通过关联OOS模板的方式来实现自动化管理MongoDB白名单的工作,关于lifecycleHook与OOS如何协作,你可以查看兄弟文章AutoScaling 通过lifecycleHook自动添加Redis白名单。
825 0