MySQL运维之 binlog_gtid_simple_recovery(GTID)-阿里云开发者社区

开发者社区> 兰春> 正文

MySQL运维之 binlog_gtid_simple_recovery(GTID)

简介:
+关注继续查看

binlog_gtid_simple_recovery 是什么

  • 官方解释
This variable controls how binary log files are iterated during the search for GTIDs when MySQL starts or restarts.
In MySQL version 5.7.5, this variable was added as simplified_binlog_gtid_recovery and in MySQL version 5.7.6 it was renamed to binlog_gtid_simple_recovery.

When binlog_gtid_simple_recovery=FALSE, the method of iterating the binary log files is:

To initialize gtid_executed, binary log files are iterated from the newest file, stopping at the first binary log that has any Previous_gtids_log_event.
All GTIDs from Previous_gtids_log_event and Gtid_log_events are read from this binary log file.
This GTID set is stored internally and called gtids_in_binlog.
The value of gtid_executed is computed as the union of this set and the GTIDs stored in the mysql.gtid_executed table.

This process could take a long time if you had a large number of binary log files without GTID events, for example created when gtid_mode=OFF.

To initialize gtid_purged, binary log files are iterated from the oldest to the newest,
stopping at the first binary log that contains either a Previous_gtids_log_event that is nonempty (that has at least one GTID)
or that has at least one Gtid_log_event. From this binary log it reads Previous_gtids_log_event.
This GTID set is subtracted from gtids_in_binlog and the result stored in the internal variable gtids_in_binlog_not_purged.
The value of gtid_purged is initialized to the value of gtid_executed, minus gtids_in_binlog_not_purged.

When binlog_gtid_simple_recovery=TRUE, which is the default in MySQL 5.7.7 and later,
the server iterates only the oldest and the newest binary log files
and the values of gtid_purged and gtid_executed are computed based only on Previous_gtids_log_event or Gtid_log_event found in these files.
This ensures only two binary log files are iterated during server restart or when binary logs are being purged.
  • 官方注意点
Note

If this option is enabled, gtid_executed and gtid_purged may be initialized incorrectly in the following situations:

The newest binary log was generated by MySQL 5.7.5 or older, and gtid_mode was ON for some binary logs but OFF for the newest binary log.

A SET GTID_PURGED statement was issued on a MySQL version prior to 5.7.7, and the binary log that was active at the time of the SET GTID_PURGED has not yet been purged.

If an incorrect GTID set is computed in either situation, it will remain incorrect even if the server is later restarted, regardless of the value of this option.
  • 个人理解与总结
1. 这个变量用于在MySQL重启或启动的时候寻找GTIDs过程中,控制binlog 如何遍历的算法?
2. 当binlog_gtid_simple_recovery=FALSE 时:
    为了初始化 gtid_executed,算法是: 从newest_binlog -> oldest_binlog 方向遍历读取,如果发现有Previous_gtids_log_event , 那么就停止遍历
    为了初始化 gtid_purged,算法是:   从oldest_binlog -> newest_binlog 方向遍历读取, 如果发现有Previous_gtids_log_event(not empty)或者 至少有一个Gtid_log_event的文件,那么就停止遍历
3. 当binlog_gtid_simple_recovery=TRUE 时:
    为了初始化 gtid_executed , 算法是: 只需要读取newest_binlog
    为了初始化 gtid_purged, 算法是: 只需要读取oldest_binlog
4. 当设置binlog_gtid_simple_recovery=TRUE , 如果MySQL版本低于5.7.7 , 可能会有gitd计算出错的可能,具体参考官方文档详细描述

根据以上解读,那么如果存在非gtid的binlog比较多的时候,会非常影响性能的。
接下来,我们就来好好测试这种场景

测试案例

重点测试non-gtid和gtid混合的情况: This process could take a long time if you had a large number of binary log files without GTID events, for example created when gtid_mode=OFF.
测试当删除binlog的时候,是如何重置gtid_purged值的

  • 环境
    MySQL5.7.13
    binlog_gtid_simple_recovery = false   => 这是重点
    GTID升级:non-GTID -> GTID 后,purge binary logs  => 这也是重点
  • binlog
-rw-r-----  1 mysql mysql        177 May  3 11:23 tjtx-126-164.000001
-rw-r-----  1 mysql mysql 1074589597 May  3 11:29 tjtx-126-164.000002
-rw-r-----  1 mysql mysql 1074589060 May  3 11:30 tjtx-126-164.000003
-rw-r-----  1 mysql mysql 1074589063 May  3 11:31 tjtx-126-164.000004
-rw-r-----  1 mysql mysql 1074589065 May  3 11:32 tjtx-126-164.000005
-rw-r-----  1 mysql mysql 1074589051 May  3 11:33 tjtx-126-164.000006
-rw-r-----  1 mysql mysql 1074589045 May  3 11:33 tjtx-126-164.000007
-rw-r-----  1 mysql mysql 1074589047 May  3 11:34 tjtx-126-164.000008
-rw-r-----  1 mysql mysql 1074589050 May  3 11:35 tjtx-126-164.000009
-rw-r-----  1 mysql mysql 1074589052 May  3 11:36 tjtx-126-164.000010
-rw-r-----  1 mysql mysql 1074589062 May  3 11:37 tjtx-126-164.000011
-rw-r-----  1 mysql mysql 1074589068 May  3 11:37 tjtx-126-164.000012
-rw-r-----  1 mysql mysql 1074589045 May  3 11:38 tjtx-126-164.000013
-rw-r-----  1 mysql mysql 1074589038 May  3 11:39 tjtx-126-164.000014
-rw-r-----  1 mysql mysql 1074589055 May  3 11:40 tjtx-126-164.000015
-rw-r-----  1 mysql mysql 1074589050 May  3 11:41 tjtx-126-164.000016
-rw-r-----  1 mysql mysql 1074589063 May  3 11:41 tjtx-126-164.000017
-rw-r-----  1 mysql mysql 1074589055 May  3 11:42 tjtx-126-164.000018
-rw-r-----  1 mysql mysql 1074589048 May  3 11:43 tjtx-126-164.000019
-rw-r-----  1 mysql mysql 1074515950 May  3 11:45 tjtx-126-164.000020
-rw-r-----  1 mysql mysql 1074589069 May  3 11:46 tjtx-126-164.000021
-rw-r-----  1 mysql mysql 1074589051 May  3 11:47 tjtx-126-164.000022
-rw-r-----  1 mysql mysql 1074589063 May  3 11:47 tjtx-126-164.000023
-rw-r-----  1 mysql mysql 1074589051 May  3 11:48 tjtx-126-164.000024
-rw-r-----  1 mysql mysql  321034919 May  3 13:53 tjtx-126-164.000025
-rw-r-----  1 mysql mysql        204 May  3 13:53 tjtx-126-164.000026
-rw-r-----  1 mysql mysql        204 May  3 13:53 tjtx-126-164.000027
-rw-r-----  1 mysql mysql       1092 May  3 13:55 tjtx-126-164.000028
-rw-r-----  1 mysql mysql        194 May  3 13:55 tjtx-126-164.000029

tjtx-126-164.000001 ~ tjtx-126-164.000028

    Previous-GTIDs
    # [empty]


tjtx-126-164.000029

    #180503 13:55:05 server id 1261261646  end_log_pos 194 CRC32 0xb77b80b7     Previous-GTIDs
    # 0923e916-3c36-11e6-82a5-ecf4bbf1f518:1-3
  • 测试开始
<master>

dba:lc> purge binary logs to 'tjtx-126-164.000005';
Query OK, 0 rows affected (1 min 14.41 sec)   --执行时间竟然长达一分钟

dba:lc> insert into t select 300;  --master的事务卡住
Query OK, 1 row affected (1 min 9.42 sec)
Records: 1  Duplicates: 0  Warnings: 0

  • strace跟踪
在从头到尾遍历binlog  ,从而再次验证了我们之前的算法理论。


63639 14:07:50.394945 read(55, "05488963387-39206410793-66801786"..., 8192) = 8192 <0.000011>
63639 14:07:50.395005 read(55, "-66498258471-55447794725-7620591"..., 8192) = 8192 <0.000010>
63639 14:07:50.395065 read(55, "7;10709822844-35491948145-283531"..., 8192) = 8192 <0.000012>
63639 14:07:50.395129 read(55, "17-05336385032;74931753923-32217"..., 8192) = 8192 <0.000011>
63639 14:07:50.395191 read(55, "053-81565945575-96536403914;8342"..., 8192) = 8192 <0.000011>
63639 14:07:50.395250 read(55, "7139-77543559499-90858749831-907"..., 8192) = 8192 <0.000010>
63639 14:07:50.395310 read(55, "07981-10898305107-65423962210-93"..., 8192) = 8192 <0.000011>
63639 14:07:50.395371 read(55, "009985-68038808770-60998915978-7"..., 8192) = 8192 <0.000010>
63639 14:07:50.395430 read(55, "3665266-98504623794-11513728759-"..., 8192) = 8192 <0.000011>
63639 14:07:50.395491 read(55, "54495717-21332716078-74081433759"..., 8192) = 8192 <0.000010>
63639 14:07:50.395550 read(55, "873221923-40252274459-8633934300"..., 8192) = 8192 <0.000010>
63639 14:07:50.395610 read(55, "2609904861-91693621073-471178324"..., 8192) = 8192 <0.000010>
63639 14:07:50.125372 open("/data/mysql.bin/tjtx-126-164.~rec~", O_RDWR|O_CREAT, 0640) = 53 <0.000039>
63639 14:07:50.125769 open("/data/mysql.bin/tjtx-126-164.index_crash_safe", O_RDWR|O_CREAT, 0640) = 55 <0.000031>
63639 14:07:50.126150 open("/data/mysql.bin/tjtx-126-164.index", O_RDWR|O_CREAT, 0640) = 3 <0.000013>

。。。。。。。。。。。。。。。。。
63639 14:07:50.126554 open("/data/mysql.bin/tjtx-126-164.000005", O_RDONLY) = 55 <0.000012>
63639 14:07:53.857069 open("/data/mysql.bin/tjtx-126-164.000006", O_RDONLY) = 55 <0.000018>
63639 14:07:57.516826 open("/data/mysql.bin/tjtx-126-164.000007", O_RDONLY) = 55 <0.000016>
63639 14:08:01.169413 open("/data/mysql.bin/tjtx-126-164.000008", O_RDONLY) = 55 <0.000018>
63639 14:08:04.815608 open("/data/mysql.bin/tjtx-126-164.000009", O_RDONLY) = 55 <0.000015>
63639 14:08:08.473808 open("/data/mysql.bin/tjtx-126-164.000010", O_RDONLY) = 55 <0.000015>
63639 14:08:12.449964 open("/data/mysql.bin/tjtx-126-164.000011", O_RDONLY) = 55 <0.000018>
63639 14:08:16.251054 open("/data/mysql.bin/tjtx-126-164.000012", O_RDONLY) = 55 <0.000019>
63639 14:08:19.686003 open("/data/mysql.bin/tjtx-126-164.000013", O_RDONLY) = 55 <0.000015>
63639 14:08:23.341291 open("/data/mysql.bin/tjtx-126-164.000014", O_RDONLY) = 55 <0.000017>
63639 14:08:27.014210 open("/data/mysql.bin/tjtx-126-164.000015", O_RDONLY) = 55 <0.000016>
63639 14:08:30.625242 open("/data/mysql.bin/tjtx-126-164.000016", O_RDONLY) = 55 <0.000016>
63639 14:08:34.192385 open("/data/mysql.bin/tjtx-126-164.000017", O_RDONLY) = 55 <0.000015>
63639 14:08:37.862750 open("/data/mysql.bin/tjtx-126-164.000018", O_RDONLY) = 55 <0.000016>
63639 14:08:41.533869 open("/data/mysql.bin/tjtx-126-164.000019", O_RDONLY) = 55 <0.000016>
63639 14:08:45.202949 open("/data/mysql.bin/tjtx-126-164.000020", O_RDONLY) = 55 <0.000017>
63639 14:08:48.792088 open("/data/mysql.bin/tjtx-126-164.000021", O_RDONLY) = 55 <0.000017>
63639 14:08:52.266700 open("/data/mysql.bin/tjtx-126-164.000022", O_RDONLY) = 55 <0.000017>
63639 14:08:55.932879 open("/data/mysql.bin/tjtx-126-164.000023", O_RDONLY) = 55 <0.000017>
63639 14:08:59.594761 open("/data/mysql.bin/tjtx-126-164.000024", O_RDONLY) = 55 <0.000015>
63639 14:09:03.256451 open("/data/mysql.bin/tjtx-126-164.000025", O_RDONLY) = 55 <0.000015>
63639 14:09:04.349108 open("/data/mysql.bin/tjtx-126-164.000026", O_RDONLY) = 55 <0.000014>
63639 14:09:04.349280 open("/data/mysql.bin/tjtx-126-164.000027", O_RDONLY) = 55 <0.000010>
63639 14:09:04.349434 open("/data/mysql.bin/tjtx-126-164.000028", O_RDONLY) = 55 <0.000010>


查看fd=55的句柄:

[root@tjtx-126-164 tmp]# ll /proc/62382/fd | grep 55
lr-x------ 1 root root 64 May  3 14:17 55 -> /data/mysql.bin/tjtx-126-164.000009
[root@tjtx-126-164 tmp]# ll /proc/62382/fd | grep 55
lr-x------ 1 root root 64 May  3 14:17 55 -> /data/mysql.bin/tjtx-126-164.000010
[root@tjtx-126-164 tmp]# ll /proc/62382/fd | grep 55
lr-x------ 1 root root 64 May  3 14:17 55 -> /data/mysql.bin/tjtx-126-164.000010
[root@tjtx-126-164 tmp]# ll /proc/62382/fd | grep 55
lr-x------ 1 root root 64 May  3 14:17 55 -> /data/mysql.bin/tjtx-126-164.000010



测试二

  • 环境

    MySQL5.7.13
    binlog_gtid_simple_recovery = true
    non-GTID -> GTID 后,purge binary logs
  • 模拟开始
dba:(none)> purge binary logs to 'tjtx-126-164.000007';
Query OK, 0 rows affected (4.06 sec)  --非常快


dba:(none)> show global variables like '%gtid%';
+----------------------------------+------------------------------------------+
| Variable_name                    | Value                                    |
+----------------------------------+------------------------------------------+
| binlog_gtid_simple_recovery      | ON                                       |
| enforce_gtid_consistency         | ON                                       |
| gtid_executed                    | 0923e916-3c36-11e6-82a5-ecf4bbf1f518:1-7 |
| gtid_executed_compression_period | 1000                                     |
| gtid_mode                        | ON                                       |
| gtid_owned                       |                                          |
| gtid_purged                      |                                          |
| session_track_gtids              | OFF                                      |
+----------------------------------+------------------------------------------+
8 rows in set (0.00 sec)

strace分析:只读取了oldest的binlog 文件

115529 14:31:31.096480 open("/data/mysql.bin/tjtx-126-164.~rec~", O_RDWR|O_CREAT, 0640) = 51 <0.000031>
115529 14:31:31.096777 open("/data/mysql.bin/tjtx-126-164.index_crash_safe", O_RDWR|O_CREAT, 0640) = 52 <0.000029>
115529 14:31:31.097111 open("/data/mysql.bin/tjtx-126-164.index", O_RDWR|O_CREAT, 0640) = 3 <0.000023>
115529 14:31:31.097502 open("/data/mysql.bin/tjtx-126-164.000007", O_RDONLY) = 52 <0.000012>


dba:(none)> purge binary logs to 'tjtx-126-164.000029';
Query OK, 0 rows affected (0.00 sec)

dba:(none)> show global variables like '%gtid%';
+----------------------------------+------------------------------------------+
| Variable_name                    | Value                                    |
+----------------------------------+------------------------------------------+
| binlog_gtid_simple_recovery      | ON                                       |
| enforce_gtid_consistency         | ON                                       |
| gtid_executed                    | 0923e916-3c36-11e6-82a5-ecf4bbf1f518:1-7 |
| gtid_executed_compression_period | 1000                                     |
| gtid_mode                        | ON                                       |
| gtid_owned                       |                                          |
| gtid_purged                      | 0923e916-3c36-11e6-82a5-ecf4bbf1f518:1-3 |  --直到000029这个binlog文件读取,才能初始化gitid_purged值,否则为空
| session_track_gtids              | OFF                                      |
+----------------------------------+------------------------------------------+
8 rows in set (0.00 sec)



算法总结

1. MySQL重启
    当binlog_gtid_simple_recovery=FALSE 时:
        为了初始化 gtid_executed,算法是: 从newest_binlog -> oldest_binlog 方向遍历读取,如果发现有Previous_gtids_log_event , 那么就停止遍历。
        为了初始化 gtid_purged,算法是:   从oldest_binlog -> newest_binlog 方向遍历读取, 如果发现有Previous_gtids_log_event(not empty)或者 至少有一个Gtid_log_event的文件,那么就停止遍历
    当binlog_gtid_simple_recovery=TRUE 时:
           为了初始化 gtid_executed , 算法是: 只需要读取newest_binlog。 如果没有,则为空
        为了初始化 gtid_purged, 算法是: 只需要读取oldest_binlog。如果没有,则为空

2. binlog rotate(expire_logs_day , purge binary logs to '' 等)
    当binlog_gtid_simple_recovery=FALSE 时:
        为了初始化 gtid_purged , 从oldest_binlog -> newest_binlog 方向遍历读取, 如果发现有Previous_gtids_log_event(not empty)或者 至少有一个Gtid_log_event的文件,那么就停止遍历
    当binlog_gtid_simple_recovery=TRUE 时:
        为了初始化 gtid_purged, 算法是: 只需要读取oldest_binlog。 如果没有,则为空

需要注意的点

  • 在线GTID升级的时候,binlog_gtid_simple_recovery = TRUE 必须打开,否则在binlog 删除的时候,会发生阻塞状况
  • 在线GTID升级的时候,尽量将非GTID的binlog备份好,然后删除掉,以免出现莫名其妙的错误

版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。

相关文章
阿里云服务器怎么设置密码?怎么停机?怎么重启服务器?
如果在创建实例时没有设置密码,或者密码丢失,您可以在控制台上重新设置实例的登录密码。本文仅描述如何在 ECS 管理控制台上修改实例登录密码。
7917 0
阿里云服务器如何登录?阿里云服务器的三种登录方法
购买阿里云ECS云服务器后如何登录?场景不同,大概有三种登录方式:
2660 0
阿里云服务器ECS远程登录用户名密码查询方法
阿里云服务器ECS远程连接登录输入用户名和密码,阿里云没有默认密码,如果购买时没设置需要先重置实例密码,Windows用户名是administrator,Linux账号是root,阿小云来详细说下阿里云服务器远程登录连接用户名和密码查询方法
10305 0
使用OpenApi弹性释放和设置云服务器ECS释放
云服务器ECS的一个重要特性就是按需创建资源。您可以在业务高峰期按需弹性的自定义规则进行资源创建,在完成业务计算的时候释放资源。本篇将提供几个Tips帮助您更加容易和自动化的完成云服务器的释放和弹性设置。
11724 0
阿里云服务器如何登录?阿里云服务器的三种登录方法
购买阿里云ECS云服务器后如何登录?场景不同,阿里云优惠总结大概有三种登录方式: 登录到ECS云服务器控制台 在ECS云服务器控制台用户可以更改密码、更换系.
11355 0
windows server 2008阿里云ECS服务器安全设置
最近我们Sinesafe安全公司在为客户使用阿里云ecs服务器做安全的过程中,发现服务器基础安全性都没有做。为了为站长们提供更加有效的安全基础解决方案,我们Sinesafe将对阿里云服务器win2008 系统进行基础安全部署实战过程! 比较重要的几部分 1.
8217 0
如何设置阿里云服务器安全组?阿里云安全组规则详细解说
阿里云安全组设置详细图文教程(收藏起来) 阿里云服务器安全组设置规则分享,阿里云服务器安全组如何放行端口设置教程。阿里云会要求客户设置安全组,如果不设置,阿里云会指定默认的安全组。那么,这个安全组是什么呢?顾名思义,就是为了服务器安全设置的。安全组其实就是一个虚拟的防火墙,可以让用户从端口、IP的维度来筛选对应服务器的访问者,从而形成一个云上的安全域。
6721 0
阿里云ECS云服务器初始化设置教程方法
阿里云ECS云服务器初始化是指将云服务器系统恢复到最初状态的过程,阿里云的服务器初始化是通过更换系统盘来实现的,是免费的,阿里云百科网分享服务器初始化教程: 服务器初始化教程方法 本文的服务器初始化是指将ECS云服务器系统恢复到最初状态,服务器中的数据也会被清空,所以初始化之前一定要先备份好。
6472 0
阿里云服务器ECS登录用户名是什么?系统不同默认账号也不同
阿里云服务器Windows系统默认用户名administrator,Linux镜像服务器用户名root
3059 0
+关注
兰春
数据库技术爱好者,专注于MySQL领域的运维与运营,擅长性能调优,系统瓶颈分析,热爱数据领域的一切
74
文章
0
问答
文章排行榜
最热
最新
相关电子书
更多
《2021云上架构与运维峰会演讲合集》
立即下载
《零基础CSS入门教程》
立即下载
《零基础HTML入门教程》
立即下载