innodb buffer pool有几个目的:
- 缓存数据--众所周知,这个占了buffer pool的大半空间
- 缓存目录--数据字典
- insert buffer
- 排序的内部结构--比如自适应hash的结构或者一些行锁
查看表的数据和索引使用情况?
SELECT engine, count(*) as TABLES,
concat(round(sum(table_rows)/1000000,2),'M') rows,
concat(round(sum(data_length)/(1024*1024*1024),2),'G') DATA,
concat(round(sum(index_length)/(1024*1024*1024),2),'G') idx,
concat(round(sum(data_length+index_length)/(1024*1024*1024),2),'G') total_size,
round(sum(index_length)/sum(data_length),2) idxfrac
FROM information_schema.TABLES
WHERE table_schema not in ('mysql', 'performance_schema', 'information_schema','test')
GROUP BY engine ORDER BY sum(data_length+index_length) DESC LIMIT 10;
+--------+--------+----------+---------+--------+------------+---------+
| engine | TABLES | rows | DATA | idx | total_size | idxfrac |
+--------+--------+----------+---------+--------+------------+---------+
| InnoDB | 71608 | 1644.51M | 130.79G | 82.76G | 213.55G | 0.63 |
+--------+--------+----------+---------+--------+------------+---------+
idxfrac这个值越低越好,举个例子,表里只有一个唯一索引的数据如下:
+--------+--------+----------+---------+--------+------------+---------+
| engine | TABLES | rows | DATA | idx | total_size | idxfrac |
+--------+--------+----------+---------+--------+------------+---------+
| InnoDB | 16 | 3120.61M | 386.59G | 58.09G | 444.68G | 0.15 |
+--------+--------+----------+---------+--------+------------+---------+
可见idxfrac可见这个值越低越好。
获取buffer pool占的page个数
select count(*) from information_schema.innodb_buffer_page;
+----------+
| count(*) |
+----------+
| 262142 |
+----------+
获取page类型:
select page_type as Page_Type,sum(data_size)/1024/1024 as Size_in_MB
from information_schema.innodb_buffer_page
group by page_type
order by Size_in_MB desc;
+-------------------+--------------+
| Page_Type | Size_in_MB |
+-------------------+--------------+
| INDEX | 158.66378689 |
| UNKNOWN | 0.00000000 |
| TRX_SYSTEM | 0.00000000 |
| SYSTEM | 0.00000000 |
| FILE_SPACE_HEADER | 0.00000000 |
| IBUF_BITMAP | 0.00000000 |
| EXTENT_DESCRIPTOR | 0.00000000 |
| ALLOCATED | 0.00000000 |
| INODE | 0.00000000 |
| BLOB | 0.00000000 |
| UNDO_LOG | 0.00000000 |
| IBUF_FREE_LIST | 0.00000000 |
| IBUF_INDEX | 0.00000000 |
+-------------------+--------------+
从这里可以看到数据和索引占了buffer pool的大部分空间。也可以看出来这里有几种重要的页类型:
- INDEX: B-Tree index
- IBUF_INDEX: Insert buffer index
- UNKNOWN: not allocated / unknown state
- TRX_SYSTEM: transaction system data
buffer pool里每个索引的使用
select table_name as Table_Name, index_name as Index_Name,count(*) as Page_Count, sum(data_size)/1024/1024 as Size_in_MB
from information_schema.innodb_buffer_page
group by table_name, index_name
order by Size_in_MB desc;
+--------------------------------------------+-----------------+------------+-------------+
| Table_Name | Index_Name | Page_Count | Size_in_MB |
+--------------------------------------------+-----------------+------------+-------------+
| `magento`.`core_url_rewrite` | PRIMARY | 2829 | 40.64266014 |
| `magento`.`core_url_rewrite` | FK_CORE_URL_... | 680 | 6.67517281 |
| `magento`.`catalog_product_entity_varchar` | PRIMARY | 449 | 6.41064930 |
| `magento`.`catalog_product_index_price` | PRIMARY | 440 | 6.29357910 |
| `magento`.`catalog_product_entity` | PRIMARY | 435 | 6.23898315 |
+--------------------------------------------+-----------------+------------+-------------+
一个典型的buffer pool使用监控
从这里图里我们可以看到buffer pool几乎是被填满的,另外预留了10%的空间用来做其他用途。
一般怎么设置buffer pool大小呢?
warm rows data size + warm indexes size (excl. clustered) + 20%
如何预热buffer pool?
在InnoDB上面执行select语句:
- 对于聚簇索引来说,大多数情况通过SELECT COUNT(*) 加载到buffer pool中了。
- 对于二级索引来说,要执行一些简单的语句来抓取全部数据,比如select from tbname where 索引的第一列。或者select from tbname force index(二级索引) where colname <>0.
另外,MySQL5.7支持动态修改buffer pool:
mysql> SET GLOBAL innodb_buffer_pool_size=size_in_bytes;
Dump & restore
在MySQL (5.6+), Percona Server (5.5.10+) or MariaDB (10.0+)可以通过以下配置把buffer pool里面的数据dump出来,并在启动的时候加载到内存中:
innodb_buffer_pool_dump_at_shutdown=ON
innodb_buffer_pool_load_at_startup=ON
参考
https://michael.bouvy.net/blog/en/2015/01/18/understanding-mysql-innodb-buffer-pool-size/
http://www.speedemy.com/mysql/17-key-mysql-config-file-settings/innodb_buffer_pool_size/