在MySQL5.7.4中，对Server层的Metadata Lock做了颠覆性的修改，在正常负载下几乎完全消除了MDL子系统部分的锁开销.
基于最新的MySQL5.7.5, 我们来看看MDL锁的具体实现思路。注意在5.7.5里，MDL子系统的代码和之前版本已经发生了非常大的差异，具体表现在：
1. 针对DDL 和DML做了区分，引入了fast path的概念，对于走fast path路径的MDL锁，无需任何读写锁操作，使用类似LOCK WORD的方式来标记赋予权限
2. 对MDL的存储采用LOCK FREE HASH的方式，这样5.6所谓的mdl分区在5.7里都全部移除了
3. THR_LOCK被移除掉了，改而全部使用MDL锁来代替，为了适应这种变化，引入新的MDL锁类型来处理类似LOCK TABLE READ这样的场景，同时例如DML饥饿处理也做了些变化
另外，鉴于MDL子系统的复杂度，本文只是做一些简单的描述，后续还会单独分篇来对其中的某些细节进行探讨。很早之前写了篇博客来介绍MDL锁的发展历程，有兴趣的可以看一下：http://mysqllover.com/?p=985
先画个图，简单的理一下各个类之间的关系
0.background
在开始之前，先了解下MDL锁划分为哪几种类型，这有助于理解mdl，并且代码的注释也非常清楚，值得一读.锁类型enum_mdl_type:
name |
简写 |
description |
| MDL_INTENTION_EXCLUSIVE | An intention exclusive metadata lock. Used only for scoped locks.Owner of this type of lock can acquire upgradable exclusive locks on individual objects.Compatible with other IX locks, but is incompatible with scoped S andX locks. | |
| MDL_SHARED | S | To be used in cases when we are interested in object metadata only and there is no intention to access object data (e.g. for stored routines or during preparing prepared statements) |
| MDL_SHARED_HIGH_PRIO | SH | high priority shared metadata lock “High priority” means that, unlike other shared locks, it is granted ignoring pending requests for exclusive locks. Intended for use incases when we only need to access metadata and not data, e.g. whenfilling an INFORMATION_SCHEMA table. Since SH lock is compatible with SNRW lock, the connection that holds SH lock lock should not try to acquire any kind of table-level or row-level lock, as this can lead to a deadlock. Moreover, after acquiring SH lock, the connection should not wait for any other resource, as it might cause starvation for X locks and a potential deadlock during upgrade of SNW or SNRW to X lock (e.g. if the upgrading connection holds the resource that is being waited for) |
| MDL_SHARED_READ | SR | A shared metadata lock for cases when there is an intention to read data from table. A connection holding this kind of lock can read table metadata and read table data (after acquiring appropriate table and row-level locks).This means that one can only acquire TL_READ, TL_READ_NO_INSERT, and similar table-level locks on table if one holds SR MDL lock on it. To be used for tables in SELECTs, subqueries, and LOCK TABLE … READ statements. |
| MDL_SHARED_WRITE | SW | A shared metadata lock for cases when there is an intention to modify (and not just read) data in the table. A connection holding SW lock can read table metadata and modify or read table data (after acquiring appropriate table and row-level locks).To be used for tables to be modified by INSERT, UPDATE, DELETE statements, but not LOCK TABLE … WRITE or DDL). Also taken by SELECT … FOR UPDATE. |
| MDL_SHARED_WRITE_LOW_PRIO | A version of MDL_SHARED_WRITE lock which has lower priority than MDL_SHARED_READ_ONLY locks. Used by DML statements modifying tables and using the LOW_PRIORITY clause. | |
| MDL_SHARED_UPGRADABLE | SU | An upgradable shared metadata lock which allows concurrent updates and reads of table data. A connection holding this kind of lock can read table metadata and read table data. It should not modify data as this lock is compatible with SRO locks.Can be upgraded to SNW, SNRW and X locks. Once SU lock is upgraded to X or SNRW lock data modification can happen freely. To be used for the first phase of ALTER TABLE. |
| MDL_SHARED_READ_ONLY | SRO | A shared metadata lock for cases when we need to read data from table and block all concurrent modifications to it (for both data and metadata). Used by LOCK TABLES READ statement. |
| MDL_SHARED_NO_WRITE | SNW | An upgradable shared metadata lock which blocks all attempts to update table data, allowing reads. A connection holding this kind of lock can read table metadata and read table data.Can be upgraded to X metadata lock. Note, that since this type of lock is not compatible with SNRW or SW lock types, acquiring appropriate engine-level locks for reading (TL_READ* for MyISAM, shared row locks in InnoDB) should be contention-free. To be used for the first phase of ALTER TABLE, when copying data between tables, to allow concurrent SELECTs from the table, but not UPDATEs. |
| MDL_SHARED_NO_READ_WRITE | SNRW | An upgradable shared metadata lock which allows other connections to access table metadata, but not data.It blocks all attempts to read or update table data, while allowing INFORMATION_SCHEMA and SHOW queries.A connection holding this kind of lock can read table metadata modify and read table data.Can be upgraded to X metadata lock. To be used for LOCK TABLES WRITE statement.Not compatible with any other lock type except S and SH. |
| MDL_EXCLUSIVE | An exclusive metadata lock. A connection holding this lock can modify both table’s metadata and data.No other type of metadata lock can be granted while this lock is held. To be used for CREATE/DROP/RENAME TABLE statements and for execution of certain phases of other DDL statements. |
enum_mdl_duration : DML duration类型
| Name | Description |
| MDL_STATEMENT | 语句级别，SQL结束后自动释放 |
| MDL_TRANSACTION | 事务提交后释放 |
| MDL_EXPLICIT | 需要显式释放锁 |
2.构建一个MDL锁请求
在尝试加锁之前，总是需要先构建一个mdl锁请求，对应的类对象为MDL_request，主要成员包括:
| enum_mdl_type type | 请求的MDL锁类型 |
| enum enum_mdl_duration duration | duration级别 |
| MDL_request *next_in_list，MDL_request **prev_in_list; | 维持当前session请求的MDL锁的链表结构 |
| MDL_ticket *ticket | 后文描述 |
| MDL_key key; | 锁请求的key一般是基于库表名或者类型 |
| init_with_sourceinit_by_key_with_source | 初始化锁请求 |
| bool is_write_lock_request() | 判断是否是写锁请求type >= MDL_SHARED_WRITE && type != MDL_SHARED_READ_ONLY |
| bool is_ddl_or_lock_tables_lock_request() | 判断是否是一个强优先级的，例如DDL, LOCK TABLE之类的请求 type >= MDL_SHARED_UPGRADABLE; |
MDL_key 用于确定一个具体的MDL锁对象，该类主要包括：
| enum enum_mdl_namespace { GLOBAL=0, SCHEMA, TABLE, FUNCTION, PROCEDURE, TRIGGER, EVENT, COMMIT, USER_LEVEL_LOCK, /* This should be the last ! */ NAMESPACE_END }; |
针对不同的对象类型，定义不同的MDL锁：TABLE: 用于table和viewFUNCTION: 用于stored functionPROCEDURE：用于stored procedureTRIGGER: trigger EVENT: event scheduler event USER_LEVEL_LOCK: 用户自定义锁，通过GET_LOCK获得 |
| mdl_key_init | 初始化key的函数，根据namespace, 库名，及对象名构建key，实际上是一个简单的字符串，字符串首个字符表示namespace类型，大概分布为：/namespace/dbname/objname |
| m_length | key值长度 |
| m_db_name_length | db名长度 |
| char m_ptr[MAX_MDLKEY_LENGTH] | 存储的key值 |
例如在执行一条简单的SELECT时候，就会在语法解析阶段将需要请求的MDL锁存储到对象TABLE_LIST::mdl_request中：
函数：TABLE_LIST *st_select_lex::add_table_to_list
if (!MY_TEST(table_options & TL_OPTION_ALIAS))
{
MDL_REQUEST_INIT(& ptr->mdl_request,
MDL_key::TABLE, ptr->db, ptr->table_name, mdl_type,
MDL_TRANSACTION);
}
注：几乎所有的mdl锁请求初始化，都是通过宏MDL_REQUEST_INIT来完成的。另一种是MDL_REQUEST_INIT_BY_KEY，实际上是对MDL_request::init_with_source 和init_by_key_with_source的封装
请求的MDL锁类型，在词法/语法解析阶段完成，参考sql_yacc.yy文件，搜m_mdl_type，可以看到针对不同的token，预先赋值好锁类型
3.请求加MDL锁
MDL锁的加锁操作，通过名为MDL_context的类来进行，该类对象挂在THD::mdl_context下面.当连接创建时就会实例化一个MDL_context对象
MDL_context类描述如下:
| try_acquire_lock(MDL_request *mdl_request) | 尝试加锁 |
| acquire_locks(MDL_request_list *requests, ulong lock_wait_timeout)acquire_lock(MDL_request *mdl_request, ulong lock_wait_timeout); | 获取MDL锁，前者可以一次操作多个MDL锁请求 |
| upgrade_shared_lock(MDL_ticket *mdl_ticket, enum_mdl_type new_type, ulong lock_wait_timeout); | 升级共享锁 |
| release_all_locks_for_name(MDL_ticket *ticket);release_lock(MDL_ticket *ticket); | |
| owns_equal_or_stronger_lock(MDL_key::enum_mdl_namespace mdl_namespace, const char *db, const char *name, enum_mdl_type mdl_type) | 判断当前session是否持有等同或更强的MDL锁 |
| MDL_wait m_wait | 当锁请求被调度或者被死锁检测模块中断时，结果被记录在这里 |
| Ticket_list m_tickets[MDL_DURATION_END]; | 所有被该连接获取到的MDL ticket，针对不同的duration级别，数组被划分成了三个链表 |
| MDL_context_owner *m_owner | MDL_context的拥有者，用来和THD解除联系 |
| bool m_needs_thr_lock_abort | 看起来只在handler接口被调用，暂时忽略 |
| bool m_force_dml_deadlock_weight | Indicates that we need to use DEADLOCK_WEIGHT_DML deadlock weight for this context and ignore the deadlock weight provided by the MDL_wait_for_subgraph object which we are waiting for |
| MDL_wait_for_subgraph *m_waiting_for | 告诉死锁检测模块当前session正在等待的mdl lock或者table define cache entry.总的来说，这是冗余的，因为这些信息可以从等待队列中获得 |
| LF_PINS *m_pins; | LOCK-FREE实现的pin, 后面单独介绍 |
| m_rand_state | State for pseudo random numbers generator (PRNG) which output is used to perform random dives into MDL_lock objects hash when searching for unused objects to free |
| find_deadlock() | 检测死锁 |
MDL ticket ，如其名，相当于一张获得锁的门票，用于代表每一个请求的mdl锁，类描述如下：
| MDL_ticket *next_in_context;MDL_ticket **prev_in_context; | 维护在当前Context中的链表结构 |
| MDL_ticket *next_in_lock;MDL_ticket **prev_in_lock; | 维护在MDL_lock中的链表结构 |
has_pending_conflicting_lock() |
检查是否有冲突的锁类型 |
| downgrade_lock(enum_mdl_type type) | 对锁降级 |
| has_stronger_or_equal_type(enum_mdl_type type) | 判断是否有等同或更强优先级的锁类型 |
| bool is_incompatible_when_granted(enum_mdl_type type)bool is_incompatible_when_waiting(enum_mdl_type type) | |
| enum enum_mdl_type m_type | mdl锁类型 |
| enum_mdl_duration m_duration; | duration类型 |
MDL_context *m_ctx |
拥有当前ticket的MDL_context |
| MDL_lock *m_lock | 当前ticket对应的MDL_lock对象 |
| bool m_is_fast_path | 用于标示是否是使用”fast path”来进行加锁，这也意味着没有被纳入MDL_lock::m_granted bitmap/list，而是简单的进行计数 |
MDL_lock也是mdl子系统的核心类之一，在获取一个MDL锁时创建，对于一个给定的名字，只会存在一个实例，并且只有锁已经被grant后才存在。你可以把它视为MDL子系统中的TABLE_SHARE:
MDL_lock类描述如下：
| Ticket_list | Ticket_list类用于维护该lock上的ticket链表，两个成员：List m_list; 存储ticket链表bitmap_t m_bitmap; 链表上ticket类型的bitmap |
| struct MDL_lock_strategy { | 辅助结构体，来处理不同的锁类型，通常有两种：1.一种称为scoped lock,包括GLOBAL, COMMIT, SCHEMA 三种namespace类型对应对象为m_scoped_lock_strategy2.另外一种称为object lock对应的对象为m_object_lock_strategy
|
| bitmap_t m_granted_incompatible[MDL_TYPE_END] | 权限矩阵，用于存储哪些已经grant的类型的锁和当前申请的不相容 |
| bitmap_t m_waiting_incompatible[4][MDL_TYPE_END] | 指明哪些等待的锁类型和当前请求的不相容。出于防止DML饿死的原因，我们需要四个数组处理不同的情况：0) in “normal” situation.1) in situation when the number of successively granted “piglet” requests exceeds the max_write_lock_count limit.2) in situation when the number of successively granted “hog” requests exceeds the max_write_lock_count limit.3) in situation when both “piglet” and “hog” counters exceed limit.
|
| fast_path_state_t m_unobtrusive_lock_increment[MDL_TYPE_END] | 用于存储unobtrusive 类型的锁请求?? m_object_lock_strategy:{ 0, 1, 1, 1ULL << 20, 1ULL << 40, 1ULL << 40, 0, 0, 0, 0, 0 } 对于每种object类型的锁： “unobtrusive” types: S, SH, SR and SW “obtrusive” types: SU, SRO, SNW, SNRW, X 这里可以理解成一个lock word, 当前的请求某种锁类型对应的值，被加到m_fast_path_state , 这也是实现FAST-PATH的核心 注释： Number of locks acquired using “fast path” are encoded in the following bits of MDL_lock::m_fast_path_state: – bits 0 .. 19 – S and SH (we don’t differentiate them once acquired) – bits 20 .. 39 – SR – bits 40 .. 59 – SW and SWLP (we don’t differentiate them once acquired) |
| bool m_is_affected_by_max_write_lock_count | 是否受到max_write_lock_count参数限制的影响m_scoped_lock_strategy: falsem_object_lock_strategy: true |
| bool (*m_needs_notification)(const MDL_ticket *ticket) | 以下均为函数指针，目的是针对两种类型不同的处理方式 m_scoped_lock_strategy: NULLm_object_lock_strategy:MDL_lock::object_lock_needs_notification |
| void (*m_notify_conflicting_locks)(MDL_context *ctx, MDL_lock *lock); | m_scoped_lock_strategy: NULLm_object_lock_strategy:MDL_lock::object_lock_notify_conflicting_locks |
| bitmap_t (*m_fast_path_granted_bitmap)(const MDL_lock &lock); | m_scoped_lock_strategy:&MDL_lock::scoped_lock_fast_path_granted_bitmap m_object_lock_strategy:MDL_lock::object_lock_fast_path_granted_bitmap
参考函数：MDL_lock::can_grant_lock |
| bool (*m_needs_connection_check)(const MDL_lock *lock); | m_scoped_lock_strategy:NULLm_object_lock_strategy:MDL_lock::object_lock_needs_connection_check 等待MDL锁时进行连接检查（MDL_context::acquire_lock） |
| };############end of MDL_lock_strategy | |
| const MDL_lock_strategy *m_strategy; | |
| static const MDL_lock_strategy m_scoped_lock_strategy;static const MDL_lock_strategy m_object_lock_strategy; | 两个核心成员！！！分别代表两种不同的策略.大多数情况下，我们只关注m_object_lock_strategy |
| MDL_key key; | 当前的key |
| bool count_piglets_and_hogs(enum_mdl_type type) | |
| uint m_obtrusive_locks_granted_waiting_count; | Number of granted or waiting lock requests of “obtrusive” type. Also includes “obtrusive” lock requests for which we about to check if they can be granted |
| Ticket_list m_granted; | 已被赋权的ticket |
| Ticket_list m_waiting | 正在等待的ticket |
| volatile fast_path_state_t m_fast_path_state | fast-path特性的核心变量，通过该变量决定了是走fast path 还是slow path，实际上这里fast_path_state_t是一个long long类型，被划分成了好几段来代表不同的含义，通过逻辑运算来获取当前MDL_lock的状态 |
| static const fast_path_state_t IS_DESTROYED= 1ULL << 62 | 标示该MDL_lock即将被销毁 |
| static const fast_path_state_t HAS_OBTRUSIVE= 1ULL << 61 | 标示该MDL_lock已经被赋予给一个OBSTRUSIVE |
| static const fast_path_state_t HAS_SLOW_PATH= 1ULL << 60 | 标示有走slow path的锁已经被grant,正在等待或者检查是否可以被grant |
MDL_map用于管理所有的MDL锁（也即MDL_lock对象），因此使用一个全局static对象mdl_locks来进行存储。 MDL_map类描述如下：
| MDL_lock *find() | 查找指定MDL_key的lock |
| MDL_lock *find_or_insert() | 查找并插入 |
| lock_object_used() | 计数器m_unused_lock_objects减1 |
| lock_object_unused(MDL_context *ctx, LF_PINS *pins) | 增加未使用的对象的计数器，如果这些对象的计数值超过某些阀值，并且unused/total超过1/4，就娶尝试释放一些对象。 释放的方式随机选取，这不同于早期LRU的淘汰方式 低水位阀值默认为1000 |
| LF_HASH m_locks | lock free的hash对象 |
| MDL_lock *m_global_lock; | 全局锁，如全局读锁 |
| MDL_lock *m_commit_lock; | 为namespace为COMMIT的锁对象预先分配的MDL_lock |
| volatile int32 m_unused_lock_objects; | 未使用的MDL_lock对象个数 |
关于这里使用到的LOCK-FREE算法，后面再单独描述。
MDL_wait表示MDL锁的等待状态
| enum enum_wait_status { EMPTY = 0, GRANTED, VICTIM, TIMEOUT, KILLED } | 等待状态值 |
| enum_wait_status timed_wait(MDL_context_owner *owner, struct timespec *abs_timeout, bool signal_timeout, const PSI_stage_info *wait_state_name) | 进入等待MDL锁的调用函数 |
MDL_context::acquire_lock流程：
Step1: 调用try_acquire_lock_impl尝试去获取mdl锁
在该函数了实现了fast-patch 和slow-path.这也是在5.7里对MDL锁的优化核心
a. 首先检查当前session的MDL_context中是否已经获取的了相同或者更强的mdl锁
MDL_context::find_ticket
如果存在，赋值给mdl_request->ticket= ticket
如果不存在，则继续
b.fix_pins())
c.创建ticket
MDL_ticket::create
d.判断是否走slow path
unobtrusive_lock_increment=
MDL_lock::get_unobtrusive_lock_increment(mdl_request);
force_slow= ! unobtrusive_lock_increment || m_needs_thr_lock_abort;
即当前的锁请求类型是否是unobtrusive的（S, SH, SR and SW），另外从代码来看，m_needs_thr_lock_abort只在通过HANDLER接口时才会被使用到.对于HANDLER接口调用，看起来总会走slow path
if (! unobtrusive_lock_increment)
materialize_fast_path_locks();
e.根据key值尝试向mdl_locks对象中查找或插入key，并返回MDL_lock对象
lock= mdl_locks.find_or_insert(m_pins, key, &pinned)
f. force_slow为false，则先走fast path
这里的判断都是无锁操作，通过MDL_lock::m_fast_path_state来判断是否有不相容的MDL锁
MDL_lock::IS_DESTROYED：表示这个MDL_lock对象被销毁了，需要重新插入(mdl_locks.find_or_insert)
MDL_lock::HAS_OBTRUSIVE: 存在Obstrusive类型的锁，需要走slow path
否则修改m_fast_path_state值，增加一定的增量值（有点类似Lock word的概念）
如果通过fast path赋予锁，无需在MDL_lock::m_granted列表中加上当前ticket
g.否则走slow path
slow path会持有读写锁来进行操作，因此相对前者会略慢点.
通过slow path进行加锁操作时，会设置m_fast_path_state变量状态值为MDL_lock::HAS_SLOW_PATH | MDL_lock::HAS_OBTRUSIVE
显而易见，如果我们所有的负载都走fast-path，那么就可以完全实现无锁的操作.
slow-path走和以前版本类似的常规检查
根据类型检查是否可以赋锁：MDL_lock::can_grant_lock
如果可以，则加入链表lock->m_granted.add_ticket(ticket)
并做一次mdl锁请求饥饿检查。
Step2: 无论是否成功，都会分配ticket，如果无法获得MDL锁，那么就进入等待状态
lock= ticket->m_lock;
lock->m_waiting.add_ticket(ticket); 当前等待的ticket会被加入到MDL_lock::m_waiting队列中
进行死锁检测
will_wait_for(ticket);
find_deadlock();
然后进入condition wait
done_waiting_for()
4. 释放MDL锁
解锁函数为：
MDL_context::release_lock(enum_mdl_duration duration, MDL_ticket *ticket)
同样这里针对fast-path 和slow-path有不同的处理.
fast-path的解锁依旧是无锁操作
slow-path则需要从m_granted中移除ticket.
TODO：
1. LOCK-FREE MDL HASH的具体实现和应用
2. MDL如何进行死锁检测
3. 如何防止锁等待饿死
