In-place update in WiredTiger-阿里云开发者社区

开发者社区> 张友东(林青)> 正文

In-place update in WiredTiger

简介:
+关注继续查看

There is a great new feature in the release note of MongoDB 3.5.12.

Faster In-place Updates in WiredTiger

This work brings improvements to in-place update workloads for users running the WiredTiger engine, especially for updates to large documents. Some workloads may see a reduction of up to 7x in disk utilization (from 24 MB/s to 3 MB/s) as well as a 20% improvement in throughput.

I thought wiredtiger has impeletementd the delta page feature introduced in the bw-tree paper, that is, writing pages that are deltas from previously written pages. But after I read the source code, I found it's a totally diffirent idea, in-place update only impacted the in-meomry and journal format, the on disk layout of data is not changed.

I will explain the core of the in-place update implementation.

MongoDB introduced mutable bson to descirbe document update as incremental(delta) update.

Mutable BSON provides classes to facilitate the manipulation of existing BSON objects or the construction of new BSON objects from scratch in an incremental fashion.

Suppose you have a very large document, see 1MB

{
   _id: ObjectId("59097118be4a61d87415cd15"),
   name: "ZhangYoudong",
   birthday: "xxxx",
   fightvalue: 100,
   xxx: .... // many other fields
}

If the fightvalue is changed from 100 to 101, you can use a DamageEvent to describe the update, it just tells you the offset、size、content(kept in another array) of the change.

struct DamageEvent {
    typedef uint32_t OffsetSizeType;
    // Offset of source data (in some buffer held elsewhere).
    OffsetSizeType sourceOffset;

    // Offset of target data (in some buffer held elsewhere).
    OffsetSizeType targetOffset;

    // Size of the damage region.
    size_t size;
};

So if you have many small changes for a document, you will have DamageEvent array, MongoDB add a new storage interface to support inserting DamageEvent array (DamageVector).

bool WiredTigerRecordStore::updateWithDamagesSupported() const {
    return true;
}

StatusWith<RecordData> WiredTigerRecordStore::updateWithDamages(
    OperationContext* opCtx,
    const RecordId& id,
    const RecordData& oldRec,
    const char* damageSource,
    const mutablebson::DamageVector& damages) {

}

WiredTiger added a new update type called WT_UPDATE_MODIFIED to support MongoDB, when a WT_UPDATE_MODIFIED update happened, wiredTiger first logged a change list which is transformed from DamageVector into journal, then kept the change list in memory associated with the original record.

When the record is read, wiredTiger will first read the original record, then apply every operation in change list, returned the final record to the client.

So the core for in-place update:

  1. WiredTiger support delta update in memory and journal, so the IO of writing journal will be greatly reduced for large document.
  2. WiredTiger's data layout is kept unchanged, so the IO of writing data is not changed.

版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。

相关文章
阿里云服务器怎么设置密码?怎么停机?怎么重启服务器?
如果在创建实例时没有设置密码,或者密码丢失,您可以在控制台上重新设置实例的登录密码。本文仅描述如何在 ECS 管理控制台上修改实例登录密码。
10015 0
使用OpenApi弹性释放和设置云服务器ECS释放
云服务器ECS的一个重要特性就是按需创建资源。您可以在业务高峰期按需弹性的自定义规则进行资源创建,在完成业务计算的时候释放资源。本篇将提供几个Tips帮助您更加容易和自动化的完成云服务器的释放和弹性设置。
12069 0
windows server 2008阿里云ECS服务器安全设置
最近我们Sinesafe安全公司在为客户使用阿里云ecs服务器做安全的过程中,发现服务器基础安全性都没有做。为了为站长们提供更加有效的安全基础解决方案,我们Sinesafe将对阿里云服务器win2008 系统进行基础安全部署实战过程! 比较重要的几部分 1.
9154 0
阿里云服务器如何登录?阿里云服务器的三种登录方法
购买阿里云ECS云服务器后如何登录?场景不同,阿里云优惠总结大概有三种登录方式: 登录到ECS云服务器控制台 在ECS云服务器控制台用户可以更改密码、更换系.
13814 0
腾讯云服务器 设置ngxin + fastdfs +tomcat 开机自启动
在tomcat中新建一个可以启动的 .sh 脚本文件 /usr/local/tomcat7/bin/ export JAVA_HOME=/usr/local/java/jdk7 export PATH=$JAVA_HOME/bin/:$PATH export CLASSPATH=.
4653 0
阿里云服务器ECS登录用户名是什么?系统不同默认账号也不同
阿里云服务器Windows系统默认用户名administrator,Linux镜像服务器用户名root
4485 0
+关注
张友东(林青)
阿里云高级技术专家
105
文章
18
问答
来源圈子
更多
阿里云数据库:帮用户承担一切数据库风险,给您何止是安心!支持关系型数据库:MySQL、SQL Server、PostgreSQL、PPAS(完美兼容Oracle)、自研PB级数据存储的分布式数据库Petadata、自研金融级云数据库OceanBase支持NoSQL数据库:MongoDB、Redis、Memcache更有褚霸、丁奇、德哥、彭立勋、玄惭、叶翔等顶尖数据库专家服务。
+ 订阅
文章排行榜
最热
最新
相关电子书
更多
《2021云上架构与运维峰会演讲合集》
立即下载
《零基础CSS入门教程》
立即下载
《零基础HTML入门教程》
立即下载