本次分享来自中国HBase技术社区第七届MeetUp成都站,分享嘉宾天引 阿里巴巴 技术专家专注在大数据领域,拥有多年分布式、高并发、大规模系统的研发与实践经验,先后参与HBase、Phoenix、Lindorm等产品的内核引擎研发,目前负责阿里上万节点的HBase As a Service的发展与落地。
分享主题:HBase2.0重新定义小对象实时存取
内容概要:小对象,特别指1K~10MB范围的数据,比如图片,短视频,文档等广泛的存在于人工智能,医疗,教育,生活分享,电子商务等领域。HBase2.0在MOB技术的加持下重新定义小对象实时存取,具有低延迟,读写强一致,检索能力强,水平易扩展等关键能力。本文介绍了MOB特性的原理与实现,以及与经典对象存储相比,MOB带来的差异性与优势。
下载链接:http://hbase.group/slides/167
参考链接:HBase2.0重新定义小对象实时存取
1.背景介绍
HBase2.0加持MOB技术,支持小对象实时存取,具有读写强一致,低延迟,高并发等特点,并兼容所有企业级特性如Snapshot,Replication。
MOB问题背景诸如以下问题:
-
IO放大:
-
多副本
-
WAL
-
Flush
-
Compaction
-
-
资源限制
-
ECS 16核64G 5Gbps
-
高效云盘140MBps
-
ECS15个挂载点
-
-
写入瓶颈
-
Compaction落后导致文件数上升,进而导致flush延迟,进而导致内存瓶颈,最终阻塞写入;同时文件数过多导致查询慢
-
假设网络IO放大系数=5,则实际可用带宽为5Gbps / 8 / 5 = 128MB/s,对于1MB对象单机能提供的TPS=128
-
2.MOB原理与背景
解决思路:降低Compaction的频率,一天或者一周做一次合并。
-
系统架构
![dd156832cce5b348e160b8af7c68c64de281a272](https://yqfile.alicdn.com/dd156832cce5b348e160b8af7c68c64de281a272.png?x-oss-process=image/resize,w_1400/format,webp)
![47bb5763adf3f517dc2e8c3c8a6c68fa313b1eff](https://yqfile.alicdn.com/47bb5763adf3f517dc2e8c3c8a6c68fa313b1eff.png?x-oss-process=image/resize,w_1400/format,webp)
![1122d64344788c5c15a3ad5212b8c8e195a59bf4](https://yqfile.alicdn.com/1122d64344788c5c15a3ad5212b8c8e195a59bf4.png?x-oss-process=image/resize,w_1400/format,webp)
![c98e4f8873e1f27e7eecf68a4138ceeafd4aca0e](https://yqfile.alicdn.com/c98e4f8873e1f27e7eecf68a4138ceeafd4aca0e.png?x-oss-process=image/resize,w_1400/format,webp)
![e84e72ed4adf9eec73ad67be8ad02dc2dd9fcec0](https://yqfile.alicdn.com/e84e72ed4adf9eec73ad67be8ad02dc2dd9fcec0.png?x-oss-process=image/resize,w_1400/format,webp)
-
MOB合并策略
-
分组合并
-
按照文件所属分区以及日期两个维度进行分区,分区内的文件进行合并称为:PartitionedMobCompactor
-
-
默认每日合并
-
按天分区,小于阈值的文件参与合并,阈值默认为1028MB
-
-
周月合并
-
周合并,按周分区,当前周文件小于阈值的参与,其它周的文件小于7*阈值的参与
-
月合并,按月分区,当前周不参与,本月周小于7*阈值参与,其它月小于28*阈值的参与
-
-
-
MOB缓存
-
文件句柄缓存
-
hbase.mob.file.cache.size = 1000
-
LRU cache
-
-
MOB对象缓存
-
复用BlockCache
-
-
-
使用方式
-
Shell访问
-
hbase> create 't1', {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 102400}
-
hbase> alter ‘t1’, {NAME => ‘f1’, IS_MOB => true, MOB_THRESHOLD => 102400}
-
-
Java API
-
HColumnDescriptor hcd = new HColumnDescriptor(“f”);
hcd.setMobEnabled(true);
hcd.setMobThreshold(102400L);
-
-
3.MOB VS. 传统对象存储
-
回顾对象存储
-
模型
-
KV 结构
-
一组KV组成的集合成为Bucket
-
缺少灵活性
-
-
API
-
支持前缀扫描,获取符合条件的Keys
-
通过Key获取对象
-
检索能力弱
-
-
访问协议
-
REST
-
简洁,语言无关
-
消耗更多的链接和网络带宽
-
-
计费模式
-
按请求次数计费
-
请求频率越高成本越高
-
Bug可能引发计费灾难
-
-
-
云HBase与对象存储对比
![d32a089b8764e637cb0fc4e4cf1773a0eb6166ec](https://yqfile.alicdn.com/d32a089b8764e637cb0fc4e4cf1773a0eb6166ec.png?x-oss-process=image/resize,w_1400/format,webp)
-
从一条SQL开始
-
用户表T:包含三个属性S1、S2、S3
属性的大小为50bytes左右,包含一个对象数据Object从100KB~10MB
S1
S2
S3
Object
查询 select Object from T where S1 = xxx and S2 > yyy and S3 < zzz
-
-
对象存储方案
-
设计逻辑表为:S1+S2+S3 => Object,将S1、S2、S3组合成一个key
-
![6bba5d7068ff04c1241a26196cce764e0945008e](https://yqfile.alicdn.com/6bba5d7068ff04c1241a26196cce764e0945008e.png?x-oss-process=image/resize,w_1400/format,webp)
-
-
优势
-
读写强一致
-
支持水平扩展
-
-
劣势
-
实时性差,一次请求要查询N次服务器
-
检索能力不足,仅支持key的前缀检索
-
当属性列增多,特别是动态增加的情况下,对象存储很难支持(key会变得非常复杂)
-
-
-
MySQL+对象存储
![fdc774c44ed1d894fda1542a67baf6dfb12cf433](https://yqfile.alicdn.com/fdc774c44ed1d894fda1542a67baf6dfb12cf433.png?x-oss-process=image/resize,w_1400/format,webp)
-
-
优势
-
检索能力强
-
支持存储结构化数据
-
-
劣势
-
实时性差,一次请求要查询N次服务器
-
读写存在不一致问题
-
不支持动态列
-
运维复杂
-
-
-
HBase MOB
![d1bf4030d6a655ace6bc616d0898f3b7f8e7c8fd](https://yqfile.alicdn.com/d1bf4030d6a655ace6bc616d0898f3b7f8e7c8fd.png?x-oss-process=image/resize,w_1400/format,webp)
-
-
优势
-
实时查询,延迟低
-
读写强一致
-
检索能力强
-
支持存储结构化数据
-
支持动态列
-
支持水平扩展
-
易于维护
-
-
-
各方案对比
![01ca3ab2eaf64cbbfeb240db753067f9b7f985a3](https://yqfile.alicdn.com/01ca3ab2eaf64cbbfeb240db753067f9b7f985a3.png?x-oss-process=image/resize,w_1400/format,webp)
4.总结与展望
HBase2.0重新定义了小对象实时存取的业务访问方式,不再是索引+对象的多次查询,提供简洁的一体化解决方案。具有访问延迟低,读写强一致,检索能力强,水平易扩展等关键能力;并且具备动态列,多版本等灵活的功能;最后一体化的解决方案简化了用户端的代码,也减少了服务端的运维成本。
后续发展包括:
-
整合OSS
-
将冷数据归档到OSS,优化成本
-
-
提升MOB合并能力
-
目前由Master执行,考虑改为分布式
-
-
独立配置
-
MOB可以采用不同于主体表的压缩,编码,块大小等配置
-
![703a1490a1db961932a35f7312a73617d02d41a6](https://yqfile.alicdn.com/703a1490a1db961932a35f7312a73617d02d41a6.png?x-oss-process=image/resize,w_1400/format,webp)
![d2833c50467ef870b668288b1018cc7f510bf47b](https://yqfile.alicdn.com/d2833c50467ef870b668288b1018cc7f510bf47b.png?x-oss-process=image/resize,w_1400/format,webp)
![46fb5f8a30283e9ab4507d31cd80302a91af0991](https://yqfile.alicdn.com/46fb5f8a30283e9ab4507d31cd80302a91af0991.png?x-oss-process=image/resize,w_1400/format,webp)
![238bfb8dd09b20ae59d81f374c5a9b39ea36c183](https://yqfile.alicdn.com/238bfb8dd09b20ae59d81f374c5a9b39ea36c183.png?x-oss-process=image/resize,w_1400/format,webp)
![398637b87ece336d72b75353a663f1076794d7d4](https://yqfile.alicdn.com/398637b87ece336d72b75353a663f1076794d7d4.png?x-oss-process=image/resize,w_1400/format,webp)
![1d076bb86dac4dc52b0272a59b09aa194730e8f4](https://yqfile.alicdn.com/1d076bb86dac4dc52b0272a59b09aa194730e8f4.png?x-oss-process=image/resize,w_1400/format,webp)
![9a42f0523f338268b9f8ed33a05b5feeef61f1a2](https://yqfile.alicdn.com/9a42f0523f338268b9f8ed33a05b5feeef61f1a2.png?x-oss-process=image/resize,w_1400/format,webp)
![0b0394cb39a03ce366f32edb59c9b39b730819f8](https://yqfile.alicdn.com/0b0394cb39a03ce366f32edb59c9b39b730819f8.png?x-oss-process=image/resize,w_1400/format,webp)
![16282fb007f1f976511de39b32bc21e37b6eb187](https://yqfile.alicdn.com/16282fb007f1f976511de39b32bc21e37b6eb187.png?x-oss-process=image/resize,w_1400/format,webp)
![f0435df655fcda871291bfeafaa06fc10d9977c8](https://yqfile.alicdn.com/f0435df655fcda871291bfeafaa06fc10d9977c8.png?x-oss-process=image/resize,w_1400/format,webp)
![86b139c07a48c97bff51031e7028b3494dfdf870](https://yqfile.alicdn.com/86b139c07a48c97bff51031e7028b3494dfdf870.png?x-oss-process=image/resize,w_1400/format,webp)
![dd8eff608de760980691aebccbe1fa34faab65c5](https://yqfile.alicdn.com/dd8eff608de760980691aebccbe1fa34faab65c5.png?x-oss-process=image/resize,w_1400/format,webp)
![259568ccec75f182d8831305a213768f9df4ef81](https://yqfile.alicdn.com/259568ccec75f182d8831305a213768f9df4ef81.png?x-oss-process=image/resize,w_1400/format,webp)
![ceecfe445cd709e2cfefac81fdf35f990e24652c](https://yqfile.alicdn.com/ceecfe445cd709e2cfefac81fdf35f990e24652c.png?x-oss-process=image/resize,w_1400/format,webp)
![53e2a3671fe1b4d8865d8d8aaeb773a518e94f02](https://yqfile.alicdn.com/53e2a3671fe1b4d8865d8d8aaeb773a518e94f02.png?x-oss-process=image/resize,w_1400/format,webp)
![2d0cf30bb8a7ba12b2db35f62feb4c95e9f3facb](https://yqfile.alicdn.com/2d0cf30bb8a7ba12b2db35f62feb4c95e9f3facb.png?x-oss-process=image/resize,w_1400/format,webp)
![b86ea6aa2a391d50b45928ac6cf4e05b14d5d298](https://yqfile.alicdn.com/b86ea6aa2a391d50b45928ac6cf4e05b14d5d298.png?x-oss-process=image/resize,w_1400/format,webp)
![f27c3e5bd5d47eba606c456808a24292fc0b894b](https://yqfile.alicdn.com/f27c3e5bd5d47eba606c456808a24292fc0b894b.png?x-oss-process=image/resize,w_1400/format,webp)
![682267e37bfb56089b4ac6152504c6479ef27f99](https://yqfile.alicdn.com/682267e37bfb56089b4ac6152504c6479ef27f99.png?x-oss-process=image/resize,w_1400/format,webp)
![be9c89ae19333e7751cdb278b32d685385110934](https://yqfile.alicdn.com/be9c89ae19333e7751cdb278b32d685385110934.png?x-oss-process=image/resize,w_1400/format,webp)
![366cbc298ee771ad2610c9fb626a399854985ec1](https://yqfile.alicdn.com/366cbc298ee771ad2610c9fb626a399854985ec1.png?x-oss-process=image/resize,w_1400/format,webp)
![bb05a8893d21322d283e650f2b124a0cf77261ca](https://yqfile.alicdn.com/bb05a8893d21322d283e650f2b124a0cf77261ca.png?x-oss-process=image/resize,w_1400/format,webp)
![469bf2cd1320ddb5617502e065c07f4b12fb9aeb](https://yqfile.alicdn.com/469bf2cd1320ddb5617502e065c07f4b12fb9aeb.png?x-oss-process=image/resize,w_1400/format,webp)
![6a37dfefb0a1221715aa9b18339747afb160fa70](https://yqfile.alicdn.com/6a37dfefb0a1221715aa9b18339747afb160fa70.png?x-oss-process=image/resize,w_1400/format,webp)
![6794808ca470276749e562b04d8646ee722aca1a](https://yqfile.alicdn.com/6794808ca470276749e562b04d8646ee722aca1a.png?x-oss-process=image/resize,w_1400/format,webp)
![1222ae217f66d0866706435aadbee5ea64507a0f](https://yqfile.alicdn.com/1222ae217f66d0866706435aadbee5ea64507a0f.png?x-oss-process=image/resize,w_1400/format,webp)
![b02bcd9c4debfc0afaba1fc3ae95392b4a9c06b7](https://yqfile.alicdn.com/b02bcd9c4debfc0afaba1fc3ae95392b4a9c06b7.png?x-oss-process=image/resize,w_1400/format,webp)
![57926f95ae733b5d0f912eaa1981f201a2f98de4](https://yqfile.alicdn.com/57926f95ae733b5d0f912eaa1981f201a2f98de4.png?x-oss-process=image/resize,w_1400/format,webp)
![aebaca46810ec4c56807e7adc1bde097992ac0fc](https://yqfile.alicdn.com/aebaca46810ec4c56807e7adc1bde097992ac0fc.png?x-oss-process=image/resize,w_1400/format,webp)
![d3f2f0da5b6761a64c7049db7719525a2c492a0c](https://yqfile.alicdn.com/d3f2f0da5b6761a64c7049db7719525a2c492a0c.gif?x-oss-process=image/resize,w_1400/format,webp)
![8f0bea83ed2c6fdd73e7c1735f6e3ba308136350](https://yqfile.alicdn.com/8f0bea83ed2c6fdd73e7c1735f6e3ba308136350.png?x-oss-process=image/resize,w_1400/format,webp)