主从不同步故障案例一

本文涉及的产品
云数据库 RDS MySQL Serverless,0.5-2RCU 50GB
简介:
原创作品,允许转载,转载时请务必以超链接形式标明文章  原始出处 、作者信息和本声明。否则将追究法律责任。 http://navyaijm.blog.51cto.com/4647068/1322059

故障原因:
开发人员在主库上修改了一张表的结构,加了一个字段,从库由于各种原因没有同步过去,导致主从同步失败。

故障处理:
1、查看在从库上查看同步状态:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting  for  master to send event
Master_Host: 10.10.100.100
Master_User: slave
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000009
Read_Master_Log_Pos: 917648
Relay_Log_File: mysqld-relay-bin.000017
Relay_Log_Pos: 778406
Relay_Master_Log_File: mysql-bin.000009
Slave_IO_Running: Yes
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB: mysql,information_schema,performance_schema, test ,mysql,information_schema,performance_schema, test
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 1054
Last_Error: Error  'Unknown column ' js_code ' in ' field list ''  on query. Default database:  'web_platform' . Query:  'update act_id set js_code=' 确保该页面的PV埋点已经上报;__tj(node,snode,cid,w, '' '' , '' '' , '' '' );说明:remark; ' where act_id=11'
Skip_Counter: 0
Exec_Master_Log_Pos: 906099
Relay_Log_Space: 790258
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 1054
Last_SQL_Error: Error  'Unknown column ' js_code ' in ' field list ''  on query. Default database:  'web_platform' . Query:  'update act_id set js_code=' 确保该页面的PV埋点已经上报;__tj(node,snode,cid,w, '' '' , '' '' , '' '' );说明:remark; ' where act_id=11'
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
1 row  in  set  (0.00 sec)

2、从上面可以看出表act_id的js_code字段在从库上没有,可以对比一下主库和从库这张表的结构:
查看主库:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
mysql> desc act_id;
+-----------------+------------------+------+-----+---------+----------------+
| Field           | Type             | Null | Key | Default | Extra          |
+-----------------+------------------+------+-----+---------+----------------+
| act_id          | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| name            | varchar(512)     | NO   |     | NULL    |                |
| fields          | varchar(512)     | NO   |     | NULL    |                |
| js_code         | text             | NO   |     | NULL    |                |
| remark          | varchar(512)     | NO   |     | NULL    |                |
| create_user     | varchar(256)     | NO   |     | NULL    |                |
| duty_user_name  | varchar(256)     | NO   |     | NULL    |                |
| duty_user_email | varchar(256)     | YES  |     | NULL    |                |
| duty_user_phone | varchar(256)     | YES  |     | NULL    |                |
| create_time     | bigint(20)       | NO   |     | NULL    |                |
| update_time     | bigint(20)       | NO   |     | NULL    |                |
+-----------------+------------------+------+-----+---------+----------------+
11 rows  in  set  (0.00 sec)

查看从库:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
mysql> desc act_id;
+-----------------+------------------+------+-----+---------+----------------+
| Field           | Type             | Null | Key | Default | Extra          |
+-----------------+------------------+------+-----+---------+----------------+
| act_id          | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| name            | varchar(512)     | NO   |     | NULL    |                |
| fields          | varchar(512)     | NO   |     | NULL    |                |
| remark          | varchar(512)     | NO   |     | NULL    |                |
| create_user     | varchar(256)     | NO   |     | NULL    |                |
| duty_user_name  | varchar(256)     | NO   |     | NULL    |                |
| duty_user_email | varchar(256)     | YES  |     | NULL    |                |
| duty_user_phone | varchar(256)     | YES  |     | NULL    |                |
| create_time     | bigint(20)       | NO   |     | NULL    |                |
| update_time     | bigint(20)       | NO   |     | NULL    |                |
+-----------------+------------------+------+-----+---------+----------------+
10 rows  in  set  (0.00 sec)

3、在正式处理故障之前要停掉主库和从库的slave进程(我的环境是做了双向同步,即主主同步)

1
2
mysql> slave stop;
Query OK, 0 rows affected (0.12 sec)

4、在从库上加缺少的字段:

1
alter table act_id add js_code text not Null AFTER fields;

5、启动从库的slave进程,查看主从状态:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
mysql> slave start;
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting  for  master to send event
Master_Host: 10.10.100.100
Master_User: slave
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000009
Read_Master_Log_Pos: 917648
Relay_Log_File: mysqld-relay-bin.000117
Relay_Log_Pos: 253
Relay_Master_Log_File: mysql-bin.000009
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB: mysql,information_schema,performance_schema, test ,mysql,information_schema,performance_schema, test
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 917648
Relay_Log_Space: 556
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
1 row  in  set  (0.00 sec)

6、查看从库的master信息,在主库上从新change

1
2
3
4
5
6
7
8
从库上:
mysql> show master status \G
*************************** 1. row ***************************
File: mysql-bin.000009
Position: 235
Binlog_Do_DB:
Binlog_Ignore_DB: mysql,information_schema,performance_schema, test ,mysql,information_schema,performance_schema, test
1 row  in  set  (0.00 sec)

主库上:

1
2
mysql> change master to master_host= "10.10.100.101" , master_port=3306, master_user= 'slave' , master_password= '123456' , master_log_file= 'mysql-bin.000009' , master_log_pos=235;
1 row  in  set  (0.00 sec)

7、对比主库和从库这两张表的结构和数据是否一样:

主库:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
mysql> show create table act_id;
+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table  | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| act_id | CREATE TABLE `act_id` (
`act_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(512) NOT NULL,
`fields` varchar(512) NOT NULL,
`js_code` text NOT NULL,
`remark` varchar(512) NOT NULL,
`create_user` varchar(256) NOT NULL,
`duty_user_name` varchar(256) NOT NULL,
`duty_user_email` varchar(256) DEFAULT NULL,
`duty_user_phone` varchar(256) DEFAULT NULL,
`create_time` bigint(20) NOT NULL,
`update_time` bigint(20) NOT NULL,
PRIMARY KEY (`act_id`),
UNIQUE KEY `act_id_UNIQUE` (`act_id`)
) ENGINE=MyISAM AUTO_INCREMENT=100 DEFAULT CHARSET=utf8 |
+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row  in  set  (0.00 sec)
mysql> desc act_id;
+-----------------+------------------+------+-----+---------+----------------+
| Field           | Type             | Null | Key | Default | Extra          |
+-----------------+------------------+------+-----+---------+----------------+
| act_id          | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| name            | varchar(512)     | NO   |     | NULL    |                |
| fields          | varchar(512)     | NO   |     | NULL    |                |
| js_code         | text             | NO   |     | NULL    |                |
| remark          | varchar(512)     | NO   |     | NULL    |                |
| create_user     | varchar(256)     | NO   |     | NULL    |                |
| duty_user_name  | varchar(256)     | NO   |     | NULL    |                |
| duty_user_email | varchar(256)     | YES  |     | NULL    |                |
| duty_user_phone | varchar(256)     | YES  |     | NULL    |                |
| create_time     | bigint(20)       | NO   |     | NULL    |                |
| update_time     | bigint(20)       | NO   |     | NULL    |                |
+-----------------+------------------+------+-----+---------+----------------+
11 rows  in  set  (0.00 sec)

从库:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
mysql> show create table act_id;
+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table  | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| act_id | CREATE TABLE `act_id` (
`act_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(512) NOT NULL,
`fields` varchar(512) NOT NULL,
`js_code` text NOT NULL,
`remark` varchar(512) NOT NULL,
`create_user` varchar(256) NOT NULL,
`duty_user_name` varchar(256) NOT NULL,
`duty_user_email` varchar(256) DEFAULT NULL,
`duty_user_phone` varchar(256) DEFAULT NULL,
`create_time` bigint(20) NOT NULL,
`update_time` bigint(20) NOT NULL,
PRIMARY KEY (`act_id`),
UNIQUE KEY `act_id_UNIQUE` (`act_id`)
) ENGINE=MyISAM AUTO_INCREMENT=100 DEFAULT CHARSET=utf8 |
+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row  in  set  (0.00 sec)
mysql> desc act_id;
+-----------------+------------------+------+-----+---------+----------------+
| Field           | Type             | Null | Key | Default | Extra          |
+-----------------+------------------+------+-----+---------+----------------+
| act_id          | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| name            | varchar(512)     | NO   |     | NULL    |                |
| fields          | varchar(512)     | NO   |     | NULL    |                |
| js_code         | text             | NO   |     | NULL    |                |
| remark          | varchar(512)     | NO   |     | NULL    |                |
| create_user     | varchar(256)     | NO   |     | NULL    |                |
| duty_user_name  | varchar(256)     | NO   |     | NULL    |                |
| duty_user_email | varchar(256)     | YES  |     | NULL    |                |
| duty_user_phone | varchar(256)     | YES  |     | NULL    |                |
| create_time     | bigint(20)       | NO   |     | NULL    |                |
| update_time     | bigint(20)       | NO   |     | NULL    |                |
+-----------------+------------------+------+-----+---------+----------------+
11 rows  in  set  (0.00 sec)

处理完成。

本文出自 “屌丝运维男” 博客,请务必保留此出处http://navyaijm.blog.51cto.com/4647068/1322059

相关实践学习
基于CentOS快速搭建LAMP环境
本教程介绍如何搭建LAMP环境,其中LAMP分别代表Linux、Apache、MySQL和PHP。
全面了解阿里云能为你做什么
阿里云在全球各地部署高效节能的绿色数据中心,利用清洁计算为万物互联的新世界提供源源不断的能源动力,目前开服的区域包括中国(华北、华东、华南、香港)、新加坡、美国(美东、美西)、欧洲、中东、澳大利亚、日本。目前阿里云的产品涵盖弹性计算、数据库、存储与CDN、分析与搜索、云通信、网络、管理与监控、应用服务、互联网中间件、移动服务、视频服务等。通过本课程,来了解阿里云能够为你的业务带来哪些帮助     相关的阿里云产品:云服务器ECS 云服务器 ECS(Elastic Compute Service)是一种弹性可伸缩的计算服务,助您降低 IT 成本,提升运维效率,使您更专注于核心业务创新。产品详情: https://www.aliyun.com/product/ecs
目录
相关文章
|
4月前
|
SQL 存储 关系型数据库
MySQL主从同步延迟原因与解决方案
MySQL主从同步延迟原因与解决方案
200 0
MySQL主从同步延迟原因与解决方案
|
6月前
|
SQL 关系型数据库 MySQL
MySQL主从同步原理
MySQL主从同步原理
|
9月前
主从同步数据的流程
主从同步数据的流程
64 0
|
11月前
|
监控 关系型数据库 MySQL
如何避免主从不同步
如何避免主从不同步
73 0
|
存储 缓存 运维
【高并发/高可用/哨兵机制/集群模式/高可用与主备切换/主从复制/断点续传】
【高并发/高可用/哨兵机制/集群模式/高可用与主备切换/主从复制/断点续传】
156 0
【高并发/高可用/哨兵机制/集群模式/高可用与主备切换/主从复制/断点续传】
|
SQL 负载均衡 网络协议
关于Linux下Mysql集群同步(主从、一主多从、主从从)部署及同步策略的一些笔记
和小伙们分享一些Mysql集群主从同步部署相关的笔记 博文内容涉及: 为什么需要mysql集群主从同步 主从同步原理 部署不同主从结构的Mysql集群 一主一从 一主多从 主从从 主从同步使用的复制模式介绍配置 食用方式:了解Linux、Mysql即可 理解不足小伙伴帮忙指正
664 0
关于Linux下Mysql集群同步(主从、一主多从、主从从)部署及同步策略的一些笔记
|
SQL 数据采集 算法
Mysql主从同步及主从同步延迟解决方案
Mysql主从同步及主从同步延迟解决方案
474 0
Mysql主从同步及主从同步延迟解决方案
|
SQL 监控 关系型数据库
|
关系型数据库 MySQL 数据库