部署复制集
由三个节点组成的 复制集 为网络故障或是其他的系统故障提供了足够的冗余。该复制集也有足够的分布式读操作的能力。复制集应该保持奇数个节点，这也就保证了 选举 可以正常的进行
用3台已有的 mongod 实例来部署一个由三个节点组成的 复制集
192.168.1.3 hadoop1.abc.com hadoop1
192.168.1.4 hadoop2.abc.com hadoop2
192.168.1.5 hadoop3.abc.com hadoop3
部署复制集的注意事项
架构
在生产环境中，我们应该将每个节点部署在独立的机器上，并使用标准的MongoDB端口 27017 。使用 bind_ip 参数来限制访问MongoDB的应用程序的地址。
若使用了异地分布式架构的复制集，请确保多数 mongod 实例节点位于主数据中心中。
连通性
确保各个节点之间可以正常通讯，且各个客户端都处于安全的可信的网络环境中。可以考虑以下事项：
-
建立虚拟的专用网络。确保各个节点之间的流量是在本地网络范围内路由的。（Establish a virtual private network. Ensure that your network topology routes all traffic between members within a single site over the local area network.）
-
配置连接限制来防止未知的客户端连接到复制集。
-
配置网络设置和防火墙规则来对将MongoDB的端口仅开放给应用程序，来让应用程序发的进出数据包可以与MongoDB正常交流。
最后请确保复制集各节点可以互相通过DNS或是主机名解析。我们需要配置DNS域名或是设置 /etc/hosts 文件来配置。
这里实验，是关闭防火墙，并把selinux设置成setenforce 0
系统环境如下：
|
1
2
3
4
5
|
[root@hadoop2 data]# cat /etc/issue
CentOS release
6.5
(Final)
Kernel \r on an \m
[root@hadoop2 data]# uname -r
2.6
.
32
-
431
.el6.x86_64
|
配置文件选项：
port = 27017
bind_ip =
dbpath =
fore = true
replSet = testrs0
rest = true
详细步骤
1、建立每个节点都建立据据目录
|
1
2
|
[root@hadoop1 ~]
# mkidr -pv /mongodb/data/
[root@hadoop1 ~]
# chown mongod.mongod /mongodb/data/
|
2
将复制集中的每个节点以适当的配置参数启动。
在每个节点上启动 mongod 并通过制定 replSet 参数来指定其复制集名，并可以指定其他需要的参数
|
1
2
3
4
5
6
|
[root@hadoop1 ~]
# vim /etc/mongod.conf
//
添加如下
#Replica Set
replSet = testrs0
或者
[root@hadoop1 ~]
# mongod --replSet "testrs0"
|
确保每个节点都有相同复制集名称
[root@hadoop1 ~]# scp /etc/mongod.conf root@hadoop2:/etc/;scp /etc/mongod.conf root@hadoop2:/etc/;
注意了，如果解决启动mongod 时，出现addr already in use错误，原因启动端口被占用
|
1
2
3
4
5
|
[root@hadoop1 data]
# mongod
2015-07-29T19:15:51.728+0800 E NETWORK [initandlisten] listen(): bind() failed errno:98 Address already
in
use
for
socket: 0.0.0.0:27017
2015-07-29T19:15:51.728+0800 E NETWORK [initandlisten] addr already
in
use
2015-07-29T19:15:51.729+0800 I STORAGE [initandlisten] exception
in
initAndListen: 29 Data directory
/data/db
not found., terminating
2015-07-29T19:15:51.729+0800 I CONTROL [initandlisten] dbexit: rc: 100
|
把端口找出来，kill掉
|
1
2
|
[root@hadoop1 ~]
# netstat -anp|more
unix 2 [ ACC ] STREAM LISTENING 15588 2174
/mongod
/tmp/mongodb-27017
.sock
|
|
1
2
3
|
[root@hadoop1 ~]
# kill 2174
[root@hadoop1 ~]
# /etc/init.d/mongod start
Starting mongod: [确定]
|
[root@hadoop1 ~]# mongo
4、初始化复制集。
//使用rs.initiate()命令，MongoDB将初始化一个由当前节点构成、拥有默认配置的复制集。
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
> rs.initiate()
{
"info2"
:
"no configuration explicitly specified -- making one"
,
"me"
:
"hadoop1.abc.com:27017"
,
"info"
:
"try querying local.system.replset to see current configuration"
,
"ok"
: 0,
"errmsg"
:
"already initialized"
,
"code"
: 23
}
> rs.status()
{
"state"
: 10,
"stateStr"
:
"REMOVED"
,
"uptime"
: 38,
"optime"
: Timestamp(1438168698, 1),
"optimeDate"
: ISODate(
"2015-07-29T11:18:18Z"
),
"ok"
: 0,
"errmsg"
:
"Our replica set config is invalid or we are not a member of it"
,
"code"
: 93
}
|
查看日志 文件
2015-07-29T20:00:45.433+0800 W NETWORK [ReplicationExecutor] Failed to connect to 192.168.1.3:27017, reason: errno:111 Connection refused
2015-07-29T20:00:45.433+0800 W REPL [ReplicationExecutor] Locally stored replica set configuration does not have a valid entry for the current node; waiting for reconfig or remote heartbeat; Got "NodeNotFound No host described in new configuration 1 for replica set testrs0 maps to this node" while validating { _id: "testrs0", version: 1, members: [ { _id: 0, host: "hadoop1.abc.com:27017", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatTimeoutSecs: 10, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 } } }
2015-07-29T20:00:45.433+0800 I REPL [ReplicationExecutor] New replica set config in use: { _id: "testrs0", version: 1, members: [ { _id: 0, host: "hadoop1.abc.com:27017", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatTimeoutSecs: 10, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 } } }
2015-07-29T20:00:45.433+0800 I REPL [ReplicationExecutor] This node is not a member of the config
2015-07-29T20:00:45.433+0800 I REPL [ReplicationExecutor] transition to REMOVED
2015-07-29T20:00:45.433+0800 I REPL [ReplicationExecutor] Starting replication applier threads
2015-07-29T20:00:49.067+0800 I NETWORK [initandlisten] connection accepted from 127.0.0.1:58852 #1 (1 connection now open)
2015-07-29T20:01:17.436+0800 I COMMAND [conn1] replSet info initiate : no configuration specified. Using a default configuration for the set
2015-07-29T20:01:17.436+0800 I COMMAND [conn1] replSet created this configuration for initiation : { _id: "testrs0", version: 1, members: [ { _id: 0, host: "hadoop1.abc.com:27017" } ] }
2015-07-29T20:01:17.436+0800 I REPL [conn1] replSetInitiate admin command received from client
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
|
[root@hadoop1 ~]
# service mongod stop
Stopping mongod: [确定]
You have new mail
in
/var/spool/mail/root
[root@hadoop1 ~]
# vim /etc/mongod.conf
#开启了
bind 127.0.0.1 把本地限制访问了
#bind 127.0.0.1
[root@hadoop1 data]
# service mongod start
Starting mongod: [确定]
> rs.initiate()
{
"info2"
:
"no configuration explicitly specified -- making one"
,
"me"
:
"hadoop1.abc.com:27017"
,
"info"
:
"try querying local.system.replset to see current configuration"
,
"ok"
: 0,
"errmsg"
:
"already initialized"
,
"code"
: 23
testrs0:PRIMARY> rs.status()
{
"set"
:
"testrs0"
,
"date"
: ISODate(
"2015-07-29T12:13:27.839Z"
),
"myState"
: 1,
"members"
: [
{
"_id"
: 0,
"name"
:
"hadoop1.abc.com:27017"
,
"health"
: 1,
"state"
: 1,
"stateStr"
:
"PRIMARY"
,
"uptime"
: 232,
"optime"
: Timestamp(1438168698, 1),
"optimeDate"
: ISODate(
"2015-07-29T11:18:18Z"
),
"electionTime"
: Timestamp(1438171776, 1),
"electionDate"
: ISODate(
"2015-07-29T12:09:36Z"
),
"configVersion"
: 1,
"self"
:
true
}
],
"ok"
: 1
}
|
5、将其他的节点加入复制集。
通过 rs.add() 来将剩下的节点加入复制集。
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
|
testrs0:PRIMARY> rs.add(
"192.168.1.4:27017"
)
{
"ok"
: 1 }
testrs0:PRIMARY> rs.status()
{
"set"
:
"testrs0"
,
"date"
: ISODate(
"2015-07-30T02:09:45.871Z"
),
"myState"
: 1,
"members"
: [
{
"_id"
: 0,
"name"
:
"hadoop1.abc.com:27017"
,
"health"
: 1,
"state"
: 1,
"stateStr"
:
"PRIMARY"
,
"uptime"
: 50410,
"optime"
: Timestamp(1438222179, 1),
"optimeDate"
: ISODate(
"2015-07-30T02:09:39Z"
),
"electionTime"
: Timestamp(1438171776, 1),
"electionDate"
: ISODate(
"2015-07-29T12:09:36Z"
),
"configVersion"
: 2,
"self"
:
true
},
{
"_id"
: 1,
"name"
:
"192.168.1.4:27017"
,
"health"
: 1,
"state"
: 2,
"stateStr"
:
"SECONDARY"
,
"uptime"
: 6,
"optime"
: Timestamp(1438222179, 1),
"optimeDate"
: ISODate(
"2015-07-30T02:09:39Z"
),
"lastHeartbeat"
: ISODate(
"2015-07-30T02:09:45.081Z"
),
"lastHeartbeatRecv"
: ISODate(
"2015-07-30T02:09:45.183Z"
),
"pingMs"
: 1,
"configVersion"
: 2
}
],
"ok"
: 1
}
|
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
|
testrs0:PRIMARY> rs.add(
"192.168.1.5:27017"
)
{
"ok"
: 1 }
testrs0:PRIMARY> rs.status()
{
"set"
:
"testrs0"
,
"date"
: ISODate(
"2015-07-30T02:28:52.382Z"
),
"myState"
: 1,
"members"
: [
{
"_id"
: 0,
"name"
:
"hadoop1.abc.com:27017"
,
"health"
: 1,
"state"
: 1,
"stateStr"
:
"PRIMARY"
,
"uptime"
: 51557,
"optime"
: Timestamp(1438223187, 1),
"optimeDate"
: ISODate(
"2015-07-30T02:26:27Z"
),
"electionTime"
: Timestamp(1438171776, 1),
"electionDate"
: ISODate(
"2015-07-29T12:09:36Z"
),
"configVersion"
: 3,
"self"
:
true
},
{
"_id"
: 1,
"name"
:
"192.168.1.4:27017"
,
"health"
: 1,
"state"
: 2,
"stateStr"
:
"SECONDARY"
,
"uptime"
: 1153,
"optime"
: Timestamp(1438223187, 1),
"optimeDate"
: ISODate(
"2015-07-30T02:26:27Z"
),
"lastHeartbeat"
: ISODate(
"2015-07-30T02:28:52.337Z"
),
"lastHeartbeatRecv"
: ISODate(
"2015-07-30T02:28:50.438Z"
),
"pingMs"
: 0,
"syncingTo"
:
"hadoop1.abc.com:27017"
,
"configVersion"
: 3
},
{
"_id"
: 2,
"name"
:
"192.168.1.5:27017"
,
"health"
: 1,
"state"
: 2,
"stateStr"
:
"SECONDARY"
,
"uptime"
: 13,
"optime"
: Timestamp(1438223187, 1),
"optimeDate"
: ISODate(
"2015-07-30T02:26:27Z"
),
"lastHeartbeat"
: ISODate(
"2015-07-30T02:28:50.437Z"
),
"lastHeartbeatRecv"
: ISODate(
"2015-07-30T02:28:50.478Z"
),
"pingMs"
: 1,
"configVersion"
: 3
}
],
"ok"
: 1
}
testrs0:PRIMARY> rs.isMaster()
{
"setName"
:
"testrs0"
,
"setVersion"
: 3,
"ismaster"
:
true
,
"secondary"
:
false
,
"hosts"
: [
"hadoop1.abc.com:27017"
,
"192.168.1.4:27017"
,
"192.168.1.5:27017"
],
"primary"
:
"hadoop1.abc.com:27017"
,
"me"
:
"hadoop1.abc.com:27017"
,
"electionId"
: ObjectId(
"55b8c280790a6c1f967f6147"
),
"maxBsonObjectSize"
: 16777216,
"maxMessageSizeBytes"
: 48000000,
"maxWriteBatchSize"
: 1000,
"localTime"
: ISODate(
"2015-07-30T02:30:18Z"
),
"maxWireVersion"
: 3,
"minWireVersion"
: 0,
"ok"
: 1
}
|
其他节点hadoop2验证一下
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
testrs0:SECONDARY> rs.isMaster()
{
"setName"
:
"testrs0"
,
"setVersion"
: 3,
"ismaster"
:
false
,
"secondary"
:
true
,
"hosts"
: [
"hadoop1.abc.com:27017"
,
"192.168.1.4:27017"
,
"192.168.1.5:27017"
],
"primary"
:
"hadoop1.abc.com:27017"
,
"me"
:
"192.168.1.4:27017"
,
"maxBsonObjectSize"
: 16777216,
"maxMessageSizeBytes"
: 48000000,
"maxWriteBatchSize"
: 1000,
"localTime"
: ISODate(
"2015-07-30T02:32:43.546Z"
),
"maxWireVersion"
: 3,
"minWireVersion"
: 0,
"ok"
: 1
}
|
6、在主节点上创建数据，从节点是否获取到
|
1
2
3
4
5
6
7
|
testrs0:PRIMARY> use testdb
switched to db testdb
testrs0:PRIMARY> db.testcoll.insert({Name:
"test"
,Age: 50,Gender:
"F"
})
WriteResult({
"nInserted"
: 1 })
testrs0:PRIMARY> db.testcoll.
find
()
{
"_id"
: ObjectId(
"55b9945b92ad0ab98483695e"
),
"Name"
:
"test"
,
"Age"
: 60,
"Gender"
:
"F"
}
{
"_id"
: ObjectId(
"55b994ce92ad0ab98483695f"
),
"Name"
:
"test"
,
"Age"
: 50,
"Gender"
:
"F"
}
|
在从节点上查询，是不可以直接查询，要使用一个命令rs.slave()把自己提升为从节点
|
1
2
3
4
5
6
|
testrs0:SECONDARY> rs.slaveOk()
testrs0:SECONDARY>
use
testdb;
switched to db testdb
testrs0:SECONDARY> db.testcoll.find()
{
"_id"
: ObjectId(
"55b9945b92ad0ab98483695e"
),
"Name"
:
"test"
,
"Age"
:
60
,
"Gender"
:
"F"
}
{
"_id"
: ObjectId(
"55b994ce92ad0ab98483695f"
),
"Name"
:
"test"
,
"Age"
:
50
,
"Gender"
:
"F"
}
|
7、让主节点hadoop1挂掉
|
1
2
|
[root@hadoop1 data]# service mongod stop
Stopping mongod:
|
在hadoop3验证一下
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
|
testrs0:PRIMARY> rs.status()
{
"set"
:
"testrs0"
,
"date"
: ISODate(
"2015-07-30T04:36:19.677Z"
),
"myState"
:
1
,
"members"
: [
{
"_id"
:
0
,
"name"
:
"hadoop1.abc.com:27017"
,
"health"
:
0
,
"state"
:
8
,
"stateStr"
:
"(not reachable/healthy)"
,
"uptime"
:
0
,
"optime"
: Timestamp(
0
,
0
),
"optimeDate"
: ISODate(
"1970-01-01T00:00:00Z"
),
"lastHeartbeat"
: ISODate(
"2015-07-30T04:36:19.503Z"
),
"lastHeartbeatRecv"
: ISODate(
"2015-07-30T04:33:18.147Z"
),
"pingMs"
:
0
,
"lastHeartbeatMessage"
:
"Failed attempt to connect to hadoop1.abc.com:27017; couldn't connect to server hadoop1.abc.com:27017 (192.168.1.3), connection attempt failed"
,
"configVersion"
: -
1
},
{
"_id"
:
1
,
"name"
:
"192.168.1.4:27017"
,
"health"
:
1
,
"state"
:
2
,
"stateStr"
:
"SECONDARY"
,
"uptime"
:
7661
,
"optime"
: Timestamp(
1438225614
,
1
),
"optimeDate"
: ISODate(
"2015-07-30T03:06:54Z"
),
"lastHeartbeat"
: ISODate(
"2015-07-30T04:36:19.335Z"
),
"lastHeartbeatRecv"
: ISODate(
"2015-07-30T04:36:19.348Z"
),
"pingMs"
:
0
,
"configVersion"
:
3
},
{
"_id"
:
2
,
"name"
:
"192.168.1.5:27017"
,
"health"
:
1
,
"state"
:
1
,
"stateStr"
:
"PRIMARY"
,
"uptime"
:
7664
,
"optime"
: Timestamp(
1438225614
,
1
),
"optimeDate"
: ISODate(
"2015-07-30T03:06:54Z"
),
"electionTime"
: Timestamp(
1438230801
,
1
),
"electionDate"
: ISODate(
"2015-07-30T04:33:21Z"
),
"configVersion"
:
3
,
"self"
:
true
}
],
"ok"
:
1
}
testrs0:PRIMARY> db.isMaster()
{
"setName"
:
"testrs0"
,
"setVersion"
:
3
,
"ismaster"
:
true
,
"secondary"
:
false
,
"hosts"
: [
"hadoop1.abc.com:27017"
,
"192.168.1.4:27017"
,
"192.168.1.5:27017"
],
"primary"
:
"192.168.1.5:27017"
,
"me"
:
"192.168.1.5:27017"
,
"electionId"
: ObjectId(
"55b9a91100e446910c89a0a3"
),
"maxBsonObjectSize"
:
16777216
,
"maxMessageSizeBytes"
:
48000000
,
"maxWriteBatchSize"
:
1000
,
"localTime"
: ISODate(
"2015-07-30T04:37:33.090Z"
),
"maxWireVersion"
:
3
,
"minWireVersion"
:
0
,
"ok"
:
1
}
testrs0:PRIMARY> db.testcoll.insert ({Name:
"tom"
,Age:
45
,Gender:
"G"
})
WriteResult({
"nInserted"
:
1
})
|
回到hadoop2查看一下数据
|
1
2
3
4
5
6
|
testrs0:SECONDARY> db.testcoll.find()
{
"_id"
: ObjectId(
"55b9942d92ad0ab98483695c"
),
"Name"
:
"test"
,
"Age"
:
60
,
"Gender"
:
"F"
}
{
"_id"
: ObjectId(
"55b9944892ad0ab98483695d"
),
"Name"
:
"test"
,
"Age"
:
60
,
"Gender"
:
"F"
}
{
"_id"
: ObjectId(
"55b9945b92ad0ab98483695e"
),
"Name"
:
"test"
,
"Age"
:
60
,
"Gender"
:
"F"
}
{
"_id"
: ObjectId(
"55b994ce92ad0ab98483695f"
),
"Name"
:
"test"
,
"Age"
:
50
,
"Gender"
:
"F"
}
{
"_id"
: ObjectId(
"55b9aa714b575261aff42f25"
),
"Name"
:
"tom"
,
"Age"
:
45
,
"Gender"
:
"G"
}
|
让hadoop1上线
|
1
2
|
[root@hadoop1 data]# service mongod start
Starting mongod:
|
再验证一下状态，但不能夺回主动权，除非让它优先级高一点。
这种场景下，它自动成为从的了
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
|
testrs0:SECONDARY> rs.status()
{
"set"
:
"testrs0"
,
"date"
: ISODate(
"2015-07-30T04:44:16.534Z"
),
"myState"
:
2
,
"syncingTo"
:
"192.168.1.4:27017"
,
"members"
: [
{
"_id"
:
0
,
"name"
:
"hadoop1.abc.com:27017"
,
"health"
:
1
,
"state"
:
2
,
"stateStr"
:
"SECONDARY"
,
"uptime"
:
165
,
"optime"
: Timestamp(
1438231153
,
1
),
"optimeDate"
: ISODate(
"2015-07-30T04:39:13Z"
),
"syncingTo"
:
"192.168.1.4:27017"
,
"configVersion"
:
3
,
"self"
:
true
},
{
"_id"
:
1
,
"name"
:
"192.168.1.4:27017"
,
"health"
:
1
,
"state"
:
2
,
"stateStr"
:
"SECONDARY"
,
"uptime"
:
164
,
"optime"
: Timestamp(
1438231153
,
1
),
"optimeDate"
: ISODate(
"2015-07-30T04:39:13Z"
),
"lastHeartbeat"
: ISODate(
"2015-07-30T04:44:16.199Z"
),
"lastHeartbeatRecv"
: ISODate(
"2015-07-30T04:44:15.824Z"
),
"pingMs"
:
0
,
"syncingTo"
:
"192.168.1.5:27017"
,
"configVersion"
:
3
},
{
"_id"
:
2
,
"name"
:
"192.168.1.5:27017"
,
"health"
:
1
,
"state"
:
1
,
"stateStr"
:
"PRIMARY"
,
"uptime"
:
164
,
"optime"
: Timestamp(
1438231153
,
1
),
"optimeDate"
: ISODate(
"2015-07-30T04:39:13Z"
),
"lastHeartbeat"
: ISODate(
"2015-07-30T04:44:16.185Z"
),
"lastHeartbeatRecv"
: ISODate(
"2015-07-30T04:44:14.902Z"
),
"pingMs"
:
0
,
"electionTime"
: Timestamp(
1438230801
,
1
),
"electionDate"
: ISODate(
"2015-07-30T04:33:21Z"
),
"configVersion"
:
3
}
],
"ok"
:
1
}
testrs0:SECONDARY> rs.slaveOk()
testrs0:SECONDARY> db.testcoll.find()
testrs0:SECONDARY>
use
testdb
switched to db testdb
testrs0:SECONDARY> db.testcoll.find()
{
"_id"
: ObjectId(
"55b9942d92ad0ab98483695c"
),
"Name"
:
"test"
,
"Age"
:
60
,
"Gender"
:
"F"
}
{
"_id"
: ObjectId(
"55b9944892ad0ab98483695d"
),
"Name"
:
"test"
,
"Age"
:
60
,
"Gender"
:
"F"
}
{
"_id"
: ObjectId(
"55b9945b92ad0ab98483695e"
),
"Name"
:
"test"
,
"Age"
:
60
,
"Gender"
:
"F"
}
{
"_id"
: ObjectId(
"55b994ce92ad0ab98483695f"
),
"Name"
:
"test"
,
"Age"
:
50
,
"Gender"
:
"F"
}
{
"_id"
: ObjectId(
"55b9aa714b575261aff42f25"
),
"Name"
:
"tom"
,
"Age"
:
45
,
"Gender"
:
"G"
}
|
7、定义优先级
使用 rs.conf() 来查看 复制集配置对象 ：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
|
testrs0:SECONDARY> rs.conf()
{
"_id"
:
"testrs0"
,
"version"
:
3
,
"members"
: [
{
"_id"
:
0
,
"host"
:
"hadoop1.abc.com:27017"
,
"arbiterOnly"
:
false
,
"buildIndexes"
:
true
,
"hidden"
:
false
,
"priority"
:
1
,
"tags"
: {
},
"slaveDelay"
:
0
,
"votes"
:
1
},
{
"_id"
:
1
,
"host"
:
"192.168.1.4:27017"
,
"arbiterOnly"
:
false
,
"buildIndexes"
:
true
,
"hidden"
:
false
,
"priority"
:
1
,
"tags"
: {
},
"slaveDelay"
:
0
,
"votes"
:
1
},
{
"_id"
:
2
,
"host"
:
"192.168.1.5:27017"
,
"arbiterOnly"
:
false
,
"buildIndexes"
:
true
,
"hidden"
:
false
,
"priority"
:
1
,
"tags"
: {
},
"slaveDelay"
:
0
,
"votes"
:
1
}
],
"settings"
: {
"chainingAllowed"
:
true
,
"heartbeatTimeoutSecs"
:
10
,
"getLastErrorModes"
: {
},
"getLastErrorDefaults"
: {
"w"
:
1
,
"wtimeout"
:
0
}
}
}
|
在hadoop3的主节点把hadoop1的优先级定义为2，让它成为主节点
将复制集配置对象复制给一个变量（如 mycfg ）。然后通过该变量对该节点设置优先级。然后通过 rs.reconfig() 来更新复制集配置。
注意，设置优先级用这个命令mycfg.members[数组中第几个节点].priority = 2
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
|
testrs0:PRIMARY> rs.conf()
{
"_id"
:
"testrs0"
,
"version"
:
3
,
"members"
: [
{
"_id"
:
0
,
"host"
:
"hadoop1.abc.com:27017"
,
"arbiterOnly"
:
false
,
"buildIndexes"
:
true
,
"hidden"
:
false
,
"priority"
:
1
,
"tags"
: {
},
"slaveDelay"
:
0
,
"votes"
:
1
},
{
"_id"
:
1
,
"host"
:
"192.168.1.4:27017"
,
"arbiterOnly"
:
false
,
"buildIndexes"
:
true
,
"hidden"
:
false
,
"priority"
:
1
,
"tags"
: {
},
"slaveDelay"
:
0
,
"votes"
:
1
},
{
"_id"
:
2
,
"host"
:
"192.168.1.5:27017"
,
"arbiterOnly"
:
false
,
"buildIndexes"
:
true
,
"hidden"
:
false
,
"priority"
:
1
,
"tags"
: {
},
"slaveDelay"
:
0
,
"votes"
:
1
}
],
"settings"
: {
"chainingAllowed"
:
true
,
"heartbeatTimeoutSecs"
:
10
,
"getLastErrorModes"
: {
},
"getLastErrorDefaults"
: {
"w"
:
1
,
"wtimeout"
:
0
}
}
}
testrs0:PRIMARY> mycfg=rs.conf()
{
"_id"
:
"testrs0"
,
"version"
:
3
,
"members"
: [
{
"_id"
:
0
,
"host"
:
"hadoop1.abc.com:27017"
,
"arbiterOnly"
:
false
,
"buildIndexes"
:
true
,
"hidden"
:
false
,
"priority"
:
1
,
"tags"
: {
},
"slaveDelay"
:
0
,
"votes"
:
1
},
{
"_id"
:
1
,
"host"
:
"192.168.1.4:27017"
,
"arbiterOnly"
:
false
,
"buildIndexes"
:
true
,
"hidden"
:
false
,
"priority"
:
1
,
"tags"
: {
},
"slaveDelay"
:
0
,
"votes"
:
1
},
{
"_id"
:
2
,
"host"
:
"192.168.1.5:27017"
,
"arbiterOnly"
:
false
,
"buildIndexes"
:
true
,
"hidden"
:
false
,
"priority"
:
1
,
"tags"
: {
},
"slaveDelay"
:
0
,
"votes"
:
1
}
],
"settings"
: {
"chainingAllowed"
:
true
,
"heartbeatTimeoutSecs"
:
10
,
"getLastErrorModes"
: {
},
"getLastErrorDefaults"
: {
"w"
:
1
,
"wtimeout"
:
0
}
}
}
testrs0:PRIMARY> mycfg.members[
0
].priority =
2
2
testrs0:PRIMARY> rs.reconfig(mycfg)
{
"ok"
:
1
}
testrs0:PRIMARY>
2015
-
07
-30T14:
34
:
44.437
+
0800
I NETWORK DBClientCursor::init call() failed
2015
-
07
-30T14:
34
:
44.439
+
0800
I NETWORK trying reconnect to
127.0
.
0.1
:
27017
(
127.0
.
0.1
) failed
2015
-
07
-30T14:
34
:
44.452
+
0800
I NETWORK reconnect
127.0
.
0.1
:
27017
(
127.0
.
0.1
) ok
testrs0:SECONDARY>
|
在hadoop1上查看，并停掉hadoop1的服务
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
|
testrs0:PRIMARY> rs.status()
{
"set"
:
"testrs0"
,
"date"
: ISODate(
"2015-07-30T06:51:11.952Z"
),
"myState"
:
1
,
"members"
: [
{
"_id"
:
0
,
"name"
:
"hadoop1.abc.com:27017"
,
"health"
:
1
,
"state"
:
1
,
"stateStr"
:
"PRIMARY"
,
"uptime"
:
7780
,
"optime"
: Timestamp(
1438238074
,
1
),
"optimeDate"
: ISODate(
"2015-07-30T06:34:34Z"
),
"electionTime"
: Timestamp(
1438238079
,
1
),
"electionDate"
: ISODate(
"2015-07-30T06:34:39Z"
),
"configVersion"
:
4
,
"self"
:
true
},
{
"_id"
:
1
,
"name"
:
"192.168.1.4:27017"
,
"health"
:
1
,
"state"
:
2
,
"stateStr"
:
"SECONDARY"
,
"uptime"
:
7780
,
"optime"
: Timestamp(
1438238074
,
1
),
"optimeDate"
: ISODate(
"2015-07-30T06:34:34Z"
),
"lastHeartbeat"
: ISODate(
"2015-07-30T06:51:11.072Z"
),
"lastHeartbeatRecv"
: ISODate(
"2015-07-30T06:51:11.375Z"
),
"pingMs"
:
0
,
"configVersion"
:
4
},
{
"_id"
:
2
,
"name"
:
"192.168.1.5:27017"
,
"health"
:
1
,
"state"
:
2
,
"stateStr"
:
"SECONDARY"
,
"uptime"
:
7780
,
"optime"
: Timestamp(
1438238074
,
1
),
"optimeDate"
: ISODate(
"2015-07-30T06:34:34Z"
),
"lastHeartbeat"
: ISODate(
"2015-07-30T06:51:11.779Z"
),
"lastHeartbeatRecv"
: ISODate(
"2015-07-30T06:51:10.299Z"
),
"pingMs"
:
0
,
"configVersion"
:
4
}
],
"ok"
:
1
}
testrs0:PRIMARY> quit()
You have
new
mail
in
/
var
/spool/mail/root
[root@hadoop1 ~]# service mongo stop
mongo: 未被识别的服务
[root@hadoop1 ~]# service mongod stop
Stopping mongod:
|
在hadoop3查看，两个节点还是会自动选举主节点的
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
|
testrs0:PRIMARY> rs.status()
{
"set"
:
"testrs0"
,
"date"
: ISODate(
"2015-07-30T06:55:18.238Z"
),
"myState"
:
1
,
"members"
: [
{
"_id"
:
0
,
"name"
:
"hadoop1.abc.com:27017"
,
"health"
:
0
,
"state"
:
8
,
"stateStr"
:
"(not reachable/healthy)"
,
"uptime"
:
0
,
"optime"
: Timestamp(
0
,
0
),
"optimeDate"
: ISODate(
"1970-01-01T00:00:00Z"
),
"lastHeartbeat"
: ISODate(
"2015-07-30T06:55:16.275Z"
),
"lastHeartbeatRecv"
: ISODate(
"2015-07-30T06:51:44.879Z"
),
"pingMs"
:
5
,
"lastHeartbeatMessage"
:
"Failed attempt to connect to hadoop1.abc.com:27017; couldn't connect to server hadoop1.abc.com:27017 (192.168.1.3), connection attempt failed"
,
"configVersion"
: -
1
},
{
"_id"
:
1
,
"name"
:
"192.168.1.4:27017"
,
"health"
:
1
,
"state"
:
2
,
"stateStr"
:
"SECONDARY"
,
"uptime"
:
16000
,
"optime"
: Timestamp(
1438238074
,
1
),
"optimeDate"
: ISODate(
"2015-07-30T06:34:34Z"
),
"lastHeartbeat"
: ISODate(
"2015-07-30T06:55:17.988Z"
),
"lastHeartbeatRecv"
: ISODate(
"2015-07-30T06:55:17.988Z"
),
"pingMs"
:
1
,
"lastHeartbeatMessage"
:
"could not find member to sync from"
,
"configVersion"
:
4
},
{
"_id"
:
2
,
"name"
:
"192.168.1.5:27017"
,
"health"
:
1
,
"state"
:
1
,
"stateStr"
:
"PRIMARY"
,
"uptime"
:
16003
,
"optime"
: Timestamp(
1438238074
,
1
),
"optimeDate"
: ISODate(
"2015-07-30T06:34:34Z"
),
"electionTime"
: Timestamp(
1438239108
,
1
),
"electionDate"
: ISODate(
"2015-07-30T06:51:48Z"
),
"configVersion"
:
4
,
"self"
:
true
}
],
"ok"
:
1
}
|
把hadoop1服务启动，因为优先级设为2，故推举它为主节点
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
testrs0:PRIMARY> rs.i
rs.initiate( rs.isMaster(
testrs0:PRIMARY> rs.isMaster()
{
"setName"
:
"testrs0"
,
"setVersion"
:
4
,
"ismaster"
:
true
,
"secondary"
:
false
,
"hosts"
: [
"hadoop1.abc.com:27017"
,
"192.168.1.4:27017"
,
"192.168.1.5:27017"
],
"primary"
:
"hadoop1.abc.com:27017"
,
"me"
:
"hadoop1.abc.com:27017"
,
"electionId"
: ObjectId(
"55b9ca84ddeeac6a93355c18"
),
"maxBsonObjectSize"
:
16777216
,
"maxMessageSizeBytes"
:
48000000
,
"maxWriteBatchSize"
:
1000
,
"localTime"
: ISODate(
"2015-07-30T06:56:28.472Z"
),
"maxWireVersion"
:
3
,
"minWireVersion"
:
0
,
"ok"
:
1
}
|
8、触发重新选举
优先级为0的节点，无权触发；仅参与选举
使用的命令rs.addArb()
9、多端口复制集
应用程序需要连接多个复制集，那么每个复制集需要有不同的名字
1）为每个节点建立必要的数据文件夹：
|
1
2
3
4
5
6
|
[root@hadoop1 ~]# mkdir -pv /srv/mongodb/rs0-
0
/srv/mongodb/rs0-
1
/srv/mongodb/rs0-
2
mkdir: 已创建目录
"/srv/mongodb"
mkdir: 已创建目录
"/srv/mongodb/rs0-0"
mkdir: 已创建目录
"/srv/mongodb/rs0-1"
mkdir: 已创建目录
"/srv/mongodb/rs0-2"
You have
new
mail
in
/
var
/spool/mail/root
|
2）启动mongod实例
第一个节点
|
1
|
[root@hadoop1 rs0-
0
]# mongod --port
27018
--dbpath /srv/mongodb/rs0-
0
--replSet rs0 --smallfiles --oplogSize
128
&
|
第二个节点
|
1
|
[root@hadoop1 ~]# mongod --port
27019
--dbpath /srv/mongodb/rs0-
1
--replSet rs0 --smallfiles --oplogSize
128
&
|
第三个节点
|
1
|
[root@hadoop1 ~]# mongod --port
27020
--dbpath /srv/mongodb/rs0-
2
--replSet rs0 --smallfiles --oplogSize
128
&
|
验证
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
|
[root@hadoop1 ~]# netstat -tlnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp
0
0
0.0
.
0.0
:
27019
0.0
.
0.0
:* LISTEN
15718
/mongod
tcp
0
0
0.0
.
0.0
:
27020
0.0
.
0.0
:* LISTEN
15785
/mongod
tcp
0
0
0.0
.
0.0
:
111
0.0
.
0.0
:* LISTEN
1081
/rpcbind
tcp
0
0
0.0
.
0.0
:
28017
0.0
.
0.0
:* LISTEN
14221
/mongod
tcp
0
0
0.0
.
0.0
:
22
0.0
.
0.0
:* LISTEN
1285
/sshd
tcp
0
0
127.0
.
0.1
:
631
0.0
.
0.0
:* LISTEN
1157
/cupsd
tcp
0
0
127.0
.
0.1
:
25
0.0
.
0.0
:* LISTEN
1361
/master
tcp
0
0
0.0
.
0.0
:
44378
0.0
.
0.0
:* LISTEN
1099
/rpc.statd
tcp
0
0
0.0
.
0.0
:
27017
0.0
.
0.0
:* LISTEN
14221
/mongod
tcp
0
0
0.0
.
0.0
:
27018
0.0
.
0.0
:* LISTEN
15640
/mongod
tcp
0
0
:::
111
:::* LISTEN
1081
/rpcbind
tcp
0
0
:::
22
:::* LISTEN
1285
/sshd
tcp
0
0
::
1
:
631
:::* LISTEN
1157
/cupsd
tcp
0
0
::
1
:
25
:::* LISTEN
1361
/master
tcp
0
0
:::
48510
:::* LISTEN
1099
/rpc.statd
|
3）指明所需连接的端口，来通过 mongo 命令连接到某个 mongod 实例
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
[root@hadoop1 mongodb]# mongo --port
27018
MongoDB shell version:
3.0
.
5
connecting to:
127.0
.
0.1
:
27018
/test
2015
-
07
-30T16:
26
:
01.442
+
0800
I NETWORK [initandlisten] connection accepted from
127.0
.
0.1
:
54185
#
1
(
1
connection now open)
Server has startup warnings:
2015
-
07
-30T16:
19
:
14.667
+
0800
I CONTROL [initandlisten] ** WARNING: You are running
this
process
as
the root user, which
is
not recommended.
2015
-
07
-30T16:
19
:
14.667
+
0800
I CONTROL [initandlisten]
2015
-
07
-30T16:
19
:
14.668
+
0800
I CONTROL [initandlisten]
2015
-
07
-30T16:
19
:
14.668
+
0800
I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled
is
'always'
.
2015
-
07
-30T16:
19
:
14.668
+
0800
I CONTROL [initandlisten] ** We suggest setting it to
'never'
2015
-
07
-30T16:
19
:
14.668
+
0800
I CONTROL [initandlisten]
2015
-
07
-30T16:
19
:
14.668
+
0800
I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag
is
'always'
.
2015
-
07
-30T16:
19
:
14.668
+
0800
I CONTROL [initandlisten] ** We suggest setting it to
'never'
2015
-
07
-30T16:
19
:
14.668
+
0800
I CONTROL [initandlisten]
>
|
最后关于
部署一个由四个节点组成的异地分布复制集。
由四个节点组成的异地分布复制集有以下两个需要注意的：
-
一个节点（如``hadoop4.abc.net``） 必须是一个 arbiter。这个节点可以在服务的任何机器上运行，或者在其他的MongoDB机器上运行。
-
我们需要决定如何分配节点。下列有3种架构类型：
在大多数情况下，由于第一种架构的易用性，我们更推荐第一种架构。
-
3个节点在Site A中，一个 优先级为0的节点 在Site B中，与此同时Site A中还要有个投票节点。
-
两个节点在Site A，两个 优先级为0的节点 在Site B ，同时一个投票节点在Site A。
-
两个节点在Site A，一个优先级为0的节点在Site B，一个优先级为0的节点在Site C，同时一个投票节点在Site A。
-
在大多数情况下，由于第一种架构的易用性，更推荐第一种架构。
复制集参考
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
|
Replication Methods
in
the mongo Shell
命令 描述
rs.add() 为复制集新增节点。
rs.addArb() 为复制集新增一个 arbiter
rs.conf() 返回复制集配置信息
rs.freeze() 防止当前节点在一段时间内选举成为主节点。
rs.help() 返回 replica
set
的命令帮助
rs.initiate() 初始化一个新的复制集。
rs.printReplicationInfo() 以主节点的视角返回复制的状态报告。
rs.printSlaveReplicationInfo() 以从节点的视角返回复制状态报告。
rs.reconfig() 通过重新应用复制集配置来为复制集更新配置。
rs.remove() 从复制集中移除一个节点。
rs.slaveOk() 为当前的连接设置 slaveOk 。不推荐使用。使用 readPref() 和 Mongo.setReadPref() 来设置 read preference 。
rs.status() 返回复制集状态信息。
rs.stepDown() 让当前的 primary 变为从节点并触发 election 。
rs.syncFrom() 设置复制集节点从哪个节点处同步数据，将会覆盖默认选取逻辑。
|
复制集数据库命令
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
命令 描述
replSetFreeze 防止当前节点在一段时间内选举成为主节点。
replSetGetStatus 返回复制集的状态报告。
replSetInitiate 初始化一个新的复制集。
replSetMaintenance 开启活关闭维护模式，维护模式将使 secondary 进入
RECOVERING 状态。
replSetReconfig 为已存在的复制集应用新的配置。
replSetStepDown 强制当前的 primary *降职*变为 secondary ，并触发选举。
replSetSyncFrom 覆盖默认的复制来源选取逻辑。
resync 强制 mongod 重新从 master 进行初始化复制。仅在主-从模式。
applyOps 内部命令应用 oplog 在现在的数据集上。
isMaster 返回该节点的角色信息，包括是否为主节点。
getoptime 内部命令，返回optime。
|
本文转自 zouqingyun 51CTO博客，原文链接：http://blog.51cto.com/zouqingyun/1679771，如需转载请自行联系原作者
