cassandra 集合类型及底层存储格式介绍-阿里云开发者社区

开发者社区> 陈江@阿里> 正文

cassandra 集合类型及底层存储格式介绍

简介: cassandra的集合类型toturist 创建带有复杂cell的user表 CREATE TABLE ks.user ( id int PRIMARY KEY, addr map, complex map
+关注继续查看

cassandra的集合类型tourist

创建带有复杂cell的user表

CREATE TABLE ks.user (
    id int PRIMARY KEY,
    addr map<text, frozen<set<text>>>,
    complex map<text, frozen<map<text, text>>>,
    listcolumn list<text>,
    setcolumn set<text>
)

插入一些数据后,查询数据如下

cassandra@cqlsh:ks> select * from user;

 id | addr                                           | complex                          | listcolumn      | setcolumn
----+------------------------------------------------+----------------------------------+-----------------+-----------------
  1 | {'bj': {'ba', 'bb'}, 'shanghai': {'sa', 'sb'}} | {'bj': {'ka': 'va', 'kb': 'vb'}} | ['a', 'b', 'c'] | {'a', 'b', 'c'}

执行bin/nodetool flush,生成sst

查看sst,文本输出

tools/bin/sstabledump /data/data2/ks/user-a92ce790a8ff11e99a3d8963a5d3f9b4/md-17-big-Data.db
[
{
  "partition" : {
    "key" : [ "1" ],
    "position" : 0
  },
  "rows" : [
    {
      "type" : "row",
      "position" : 233,
      "liveness_info" : { "tstamp" : "2019-07-18T02:43:36.011497Z" },
      "cells" : [
        { "name" : "addr", "deletion_info" : { "marked_deleted" : "2019-07-18T02:43:36.011496Z", "local_delete_time" : "2019-07-18T02:43:36Z" } },
        { "name" : "addr", "path" : [ "bj" ], "value" : ["ba", "bb"] },
        { "name" : "addr", "path" : [ "shanghai" ], "value" : ["sa", "sb"] },
        { "name" : "complex", "deletion_info" : { "marked_deleted" : "2019-07-18T02:47:55.888562Z", "local_delete_time" : "2019-07-18T02:47:55Z" } },
        { "name" : "complex", "path" : [ "bj" ], "value" : {"ka": "va", "kb": "vb"}, "tstamp" : "2019-07-18T02:47:55.888563Z" },
        { "name" : "listcolumn", "deletion_info" : { "marked_deleted" : "2019-07-18T02:56:09.386468Z", "local_delete_time" : "2019-07-18T02:56:09Z" } },
        { "name" : "listcolumn", "path" : [ "982493f0-a907-11e9-9a3d-8963a5d3f9b4" ], "value" : "a", "tstamp" : "2019-07-18T02:56:09.386469Z" },
        { "name" : "listcolumn", "path" : [ "982493f1-a907-11e9-9a3d-8963a5d3f9b4" ], "value" : "b", "tstamp" : "2019-07-18T02:56:09.386469Z" },
        { "name" : "listcolumn", "path" : [ "982493f2-a907-11e9-9a3d-8963a5d3f9b4" ], "value" : "c", "tstamp" : "2019-07-18T02:56:09.386469Z" },
        { "name" : "setcolumn", "deletion_info" : { "marked_deleted" : "2019-07-18T02:55:03.280578Z", "local_delete_time" : "2019-07-18T02:55:03Z" } },
        { "name" : "setcolumn", "path" : [ "a" ], "value" : "", "tstamp" : "2019-07-18T02:55:03.280579Z" },
        { "name" : "setcolumn", "path" : [ "b" ], "value" : "", "tstamp" : "2019-07-18T02:55:03.280579Z" },
        { "name" : "setcolumn", "path" : [ "c" ], "value" : "", "tstamp" : "2019-07-18T02:55:03.280579Z" }
      ]
    }
  ]
}

底层的集合是通过cellName+path唯一标记一个元素的。
重点看下addr及complex列, 这两列是嵌套map
{ "name" : "complex", "path" : [ "bj" ], "value" : {"ka": "va", "kb": "vb"}, "tstamp" : "2019-07-18T02:47:55.888563Z" },
但对于子map frozen<map<text, text>> 基本上是当做blob存储的,不能操作map中的子元素,这也是frozen语义。

c*是无主架构,可以多node并发写同一个集合,那如何解决冲突?答案是底层最小存储单元并不是cell,而是cell+path唯一标记的element,依赖于cell&path做单元合并的,以cell timestamp最新作为最终值
删除setcolumn中一个元素

cassandra@cqlsh:ks> update user set setcolumn = setcolumn - {'a'} where id =1;

flush后,查看刚生成的sstable, setcolumn.a写入了一个delete_info

[root@Cassandra8c32GTest005 cassandra]# tools/bin/sstabledump /data/data2/ks/user-a92ce790a8ff11e99a3d8963a5d3f9b4/md-19-big-Data.db
[
{
  "partition" : {
    "key" : [ "1" ],
    "position" : 0
  },
  "rows" : [
    {
      "type" : "row",
      "position" : 27,
      "cells" : [
        { "name" : "setcolumn", "path" : [ "a" ], "deletion_info" : { "local_delete_time" : "2019-07-18T10:03:17Z" },
          "tstamp" : "2019-07-18T10:03:17.038519Z"
        }
      ]
    }
  ]
}
]

做下手工merge,可以发现setcolumn.a value没了,写入了delete_info

bin/nodetool compact
tools/bin/sstabledump /data/data2/ks/user-a92ce790a8ff11e99a3d8963a5d3f9b4/md-22-big-Data.db
[
  {
    "partition" : {
      "key" : [ "1" ],
      "position" : 0
    },
    "rows" : [
      {
        "type" : "row",
        "position" : 236,
        "liveness_info" : { "tstamp" : "2019-07-18T02:43:36.011497Z" },
        "cells" : [
          { "name" : "addr", "deletion_info" : { "marked_deleted" : "2019-07-18T02:43:36.011496Z", "local_delete_time" : "2019-07-18T02:43:36Z" } },
          { "name" : "addr", "path" : [ "bj" ], "value" : ["ba", "bb"] },
          { "name" : "addr", "path" : [ "shanghai" ], "value" : ["sa", "sb"] },
          { "name" : "complex", "deletion_info" : { "marked_deleted" : "2019-07-18T02:47:55.888562Z", "local_delete_time" : "2019-07-18T02:47:55Z" } },
          { "name" : "complex", "path" : [ "bj" ], "value" : {"ka": "va", "kb": "vb"}, "tstamp" : "2019-07-18T02:47:55.888563Z" },
          { "name" : "listcolumn", "deletion_info" : { "marked_deleted" : "2019-07-18T02:56:09.386468Z", "local_delete_time" : "2019-07-18T02:56:09Z" } },
          { "name" : "listcolumn", "path" : [ "982493f0-a907-11e9-9a3d-8963a5d3f9b4" ], "value" : "a", "tstamp" : "2019-07-18T02:56:09.386469Z" },
          { "name" : "listcolumn", "path" : [ "982493f1-a907-11e9-9a3d-8963a5d3f9b4" ], "value" : "b", "tstamp" : "2019-07-18T02:56:09.386469Z" },
          { "name" : "listcolumn", "path" : [ "982493f2-a907-11e9-9a3d-8963a5d3f9b4" ], "value" : "c", "tstamp" : "2019-07-18T02:56:09.386469Z" },
          { "name" : "setcolumn", "deletion_info" : { "marked_deleted" : "2019-07-18T02:55:03.280578Z", "local_delete_time" : "2019-07-18T02:55:03Z" } },
          { "name" : "setcolumn", "path" : [ "a" ], "deletion_info" : { "local_delete_time" : "2019-07-18T10:03:17Z" },
            "tstamp" : "2019-07-18T10:03:17.038519Z"
          },
          { "name" : "setcolumn", "path" : [ "b" ], "value" : "", "tstamp" : "2019-07-18T02:55:03.280579Z" },
          { "name" : "setcolumn", "path" : [ "c" ], "value" : "", "tstamp" : "2019-07-18T02:55:03.280579Z" }
        ]
      }
    ]
  }

试试删除setcolumn整列

update user set setcolumn = null where id =1;

刷sst,执行nodetool compact, 使用dump工具查看,setcolumn之前的子元素全部消失了。

tools/bin/sstabledump  /data/data2/ks/user-a92ce790a8ff11e99a3d8963a5d3f9b4/md-27-big-Data.db
[
  {
    "partition" : {
      "key" : [ "1" ],
      "position" : 0
    },
    "rows" : [
      {
        "type" : "row",
        "position" : 210,
        "liveness_info" : { "tstamp" : "2019-07-18T02:43:36.011497Z" },
        "cells" : [
          { "name" : "addr", "deletion_info" : { "marked_deleted" : "2019-07-18T02:43:36.011496Z", "local_delete_time" : "2019-07-18T02:43:36Z" } },
          { "name" : "addr", "path" : [ "bj" ], "value" : ["ba", "bb"] },
          { "name" : "addr", "path" : [ "shanghai" ], "value" : ["sa", "sb"] },
          { "name" : "complex", "deletion_info" : { "marked_deleted" : "2019-07-18T02:47:55.888562Z", "local_delete_time" : "2019-07-18T02:47:55Z" } },
          { "name" : "complex", "path" : [ "bj" ], "value" : {"ka": "va", "kb": "vb"}, "tstamp" : "2019-07-18T02:47:55.888563Z" },
          { "name" : "listcolumn", "deletion_info" : { "marked_deleted" : "2019-07-18T02:56:09.386468Z", "local_delete_time" : "2019-07-18T02:56:09Z" } },
          { "name" : "listcolumn", "path" : [ "982493f0-a907-11e9-9a3d-8963a5d3f9b4" ], "value" : "a", "tstamp" : "2019-07-18T02:56:09.386469Z" },
          { "name" : "listcolumn", "path" : [ "982493f1-a907-11e9-9a3d-8963a5d3f9b4" ], "value" : "b", "tstamp" : "2019-07-18T02:56:09.386469Z" },
          { "name" : "listcolumn", "path" : [ "982493f2-a907-11e9-9a3d-8963a5d3f9b4" ], "value" : "c", "tstamp" : "2019-07-18T02:56:09.386469Z" },
          { "name" : "setcolumn", "deletion_info" : { "marked_deleted" : "2019-07-18T10:11:06.966534Z", "local_delete_time" : "2019-07-18T10:11:06Z" } }
        ]
      }
    ]
  }

总结

cassandra 宽表模型是将列打平存储成一个个cell,对于集合类型,相当于把cell再打平成path存储,整个表格相当于是一个双层结构。同时集合cell有自己的deleteTime,下层的path也有自己的deleteTime, ts等。

钉钉群交流

为了营造一个开放的 Cassandra 技术交流,我们建立了微信群和钉钉群,为广大用户提供专业的技术分享及问答,定期在国内开展线下技术沙龙,专家技术直播,欢迎大家加入。
image

钉钉群入群链接:https://c.tb.cn/F3.ZRTY0o

版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。

相关文章
SpringBoot-RabbitMQ01-入门介绍
AMQP (Advanced Message Queuing Protocol ,高级消息队列协议)是 个线路层的协议规范,而不是 API 规范(例如 JMS )。由于 AMQP 是一个线路层协议规范,因此它天然就是跨平台的,就像 SMTP HTTP 等协议 样,只要开发者按照规范的格式发送数据,任何平台都可以通过 AMQP进行消息交互。像目前流行的 StormMQ RabbitMQ 等都实现了 AMQP. ———————————————— 版权声明:本文为CSDN博主「波波烤鸭」的原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接及本声明。 原文链接:https://bl
5 0
SpringBoot整合SpringDataRedis
本文介绍下SpringBoot如何整合SpringDataRedis框架的,SpringDataRedis具体的内容在前面已经介绍过了,可自行参考
5 0
SpringDataRedis介绍
实际项目中我们一般都会使用Redis来作为我们的缓存组件,往往又会和Spring一块使用,虽然Redis官方提供的有Jedis等客户端工具,但是使用的时候还是有些不方便,这时SpringDataRedis出现了。
4 0
第一次胜过MobileNet的二值神经网络,-1与+1的三年艰苦跋涉
近日,苹果以 2 亿美元左右价格收购初创公司 Xnor.ai 的消息引起了社区极大的关注。作为一家以二值神经网络 Xnor-net 起家的人工智能初创公司,Xnor.ai 被苹果高价收购,也许预示着低功耗、高效能的二值神经网络技术将开启广阔的应用前景。
4 0
云原生数据仓库AnalyticDB支撑双11,大幅提升分析实时性和用户体验
2021年双十一刚刚落幕,已连续多年稳定支持双十一大促的云原生数据仓库AnalyticDB,今年双十一期间仍然一如既往的稳定。除了稳定顺滑的基本盘之外,AnalyticDB还有什么亮点呢?下面我们来一一揭秘。
5 0
SpringBoot【SpringMVC+mybatis完成CRUD案例】
本文介绍SpringBoot整合SpringMVC和mybatis在IDEA中通过maven项目来对user表数据完成CRUD操作,前端使用Thymeleaf来实现。
6 0
面试:Java基础 3
面试:Java基础 3
4 0
Java实现链表结构
Java实现链表结构
4 0
SpringBoot-RabbitMQ02-原理介绍
本文来介绍下RabbitMQ的工作原理。在具体介绍之前最好对java的观察者模式有所了解,那么对于消息中间件就非常好理解了
6 0
+关注
陈江@阿里
从业于BAT等公有云部门,分布式存储领域专家
21
文章
0
问答
来源圈子
更多
文章排行榜
最热
最新
相关电子书
更多
文娱运维技术
立即下载
《SaaS模式云原生数据仓库应用场景实践》
立即下载
《看见新力量:二》电子书
立即下载