开发者社区> 芷沁> 正文
阿里云
为了无法计算的价值
打开APP
阿里云APP内打开

Implementing a Highly-Compressed Data Storage

简介: Alibaba Cloud ApsaraDB for RDS for MySQL supports the TokuDB engine to store data that is compressed to 5 to 10 times smaller than its original size.
+关注继续查看

ST_007

1. MySQL Big Data Storage - Compression up to 10 times smaller

Alibaba Cloud ApsaraDB for RDS for MySQL supports the TokuDB engine to store data that is compressed to 5 to 10 times smaller than its original size. It also supports highly concurrent writes through the caching of intermediate nodes.

TokuDB is an optional storage engine for RDS for MySQL. With a disk-optimized index structure Fractal Tree, TokuDB's intermediate nodes can cache data processing requests (insert/update/delete/on-line add index/on-line add column), improving the performance of high-concurrency writes by three to nine times. The node size is 4 MB (configurable), and data can be compressed by 5 to 10 times through a variety of compression algorithms such as zlib/quicklz/lzma/zstd/snappy. TokuDB also supports multiple versions of MVCC and the four isolation levels UR, RC, RR, and Serializable.

In addition to features found in the Community Edition, the source code team for RDS for MySQL also implemented a number of customized optimizations to respond to common use cases:

• Hot backup
TokuDB Community Edition does not provide hot backup. RDS for MySQL enables a hot backup solution based on the TokuDB internal checkpoint mechanism by copying the binlog, redo log, and data files. RDS for MySQL also obtains the TokuDB checkpoint lock to prevent the sharp checkpoint issue during the hot backup process.

• Improve query response on the client
The source code version provided by RDS limits TokuDB to performing sharp checkpoint once every second to avoid using too much disk bandwidth, which can result in query response failures on the client.

• Set buffer pool ratio
RDS provides a parameter tokudb_buffer_pool_ratio for you to set the percentage of memory occupied by the TokuDB engine buffer pool within the range [0,100]. The minimum TokuDB buffer pool size is 64 MB, and the minimum InnoDB buffer pool size is 64 MB (V5.6) and 128 MB (V5.7) to meet the InnoDB/TokuDB initialization requirements.

Switch RDS for MySQL to the TokuDB engine in three steps

1.Set the "loose_tokudb_buffer_pool_ratio", that is, the proportion of the TokuDB and InnoDB shared cache occupied by TokuDB.

select sum(data_length) into @all_size from information_schema.tables where engine='innodb';

select sum(data_length) into @change_size from information_schema.tables where engine='innodb' and concat(table_schema, '.', table_name) in ('XX.XXXX', 'XX.XXXX', 'XX.XXXX');

select round(@change_size/@all_size*100);

2.Restart the instance.

3.Alter the storage engine.

ALTER TABLE XX.XXXX ENGINE=TokuDB

Specifically, "XXX.XXXX" refers to the name of the database and table to be altered to the TokuDB storage engine.

2. MySQL MaxCompute

Alibaba Cloud provides the MaxCompute service for storage and calculation of batch structured data. The service offers mass data warehouse solutions and analytical modeling services for big data. You can import RDS data into MaxCompute through the Data Integration service and simple settings on the interface to achieve large-scale data computing.

29_1

版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。

相关文章
排序算法之六——希尔排序
排序算法之六——希尔排序
42 0
排序:希尔排序(算法)
(注:如果想更好的理解希尔排序,请先看看我的上一篇博客插入排序,希望会对你有帮助。)
136 0
排序算法:希尔排序
排序算法:希尔排序
29 0
十大排序算法——希尔排序
十大排序算法——希尔排序
44 0
排序算法:希尔排序
希尔排序法(缩小增量法) 属于插入类排序,是将整个无序列分割成若干小的子序列分别进行插入排序的方法。 //shell排序 /* 基本思想:将序列分成几类,在类里将它分成几个小组,再让它在小组内进行排序,依次重复直至排序成功 1,找出间距gap(分类)最后间距分类必须为1...
641 0
<转>CProcessData : A template class to ease up SendMessage calls across processes
Overview CProcessData is a template class that makes it easy to use data allocated in a different process, and is useful when making inter-process SendMessage/PostMessage calls.
824 0
+关注
芷沁
https://www.alibabacloud.com/blog/
417
文章
1
问答
文章排行榜
最热
最新
相关电子书
更多
低代码开发师(初级)实战教程
立即下载
阿里巴巴DevOps 最佳实践手册
立即下载
冬季实战营第三期:MySQL数据库进阶实战
立即下载