ClickHouse Keeper: 一个用 C++ 编写的 ZooKeeper 替代品

简介: ClickHouse Keeper: 一个用 C++ 编写的 ZooKeeper 替代品介绍




ClickHouseClickHouseKeeperClickHouseZooKeeperClickHouse使

ClickHouseKeeper使ClickHouseKeeper使ClickHouseKeeperZooKeeper使ZooKeeper1/46


ClickHouseZooKeeper广使API

ClickHouseZooKeeperJavaC++使使ZooKeeperClickHouseKeeper

ClickHouseKeeperZooKeeper

  • ClickHouseKeeperC++JavaClickHouse

  • ZooKeeper1MB
  • ZXIDZooKeeper20亿
  • 使
  • ClickHouseKeeperZooKeeper-线quorum_readsClickHouseKeeper线
  • ClickHouseKeeper使

ClickHouseKeeper20212ClickHouseJepsen-6

ClickHouseKeeper20225ClickHouseCloud

ClickHouseKeeperKeeper


ClickHouse使

ClickHouseKeeper

  • Keepershared-nothingClickHouse
  • MergetreeKeeperblock-hash-sums
  • Keeperpartpartmutation
  • KeeperKeeperMap使使Keeper线
  • ClickHouse
  • KafkaConnectSink使
  • KeeperS3Queue
  • DatabaseKeeper
  • KeeperONCLUSTER
  • UDFKeeper
  • 访Keeper
  • KeeperClickHouseCloud


Keeper

ClickHouseCloudKeeperWikiStat3ClickHouseCloud30CPU120GBRAM使ClickHouseKeeper3Keeper3CPU2GBRAM

10074046亿ClickHouseClickHouse使107GB

0 rows inset. Elapsed:101.208 sec. Processed 4.64 billion rows,40.58 GB (45.86 million rows/s.,400.93 MB/s.)Peak memory usage:107.75 GiB.

Part

3ClickHouse240partpart1934100MiB46亿

┌─parts──┬─rows_avg──────┬─size_avg───┬─rows_total───┐
│ 240.00 │ 19.34 million │ 108.89 MiB │ 4.64 billion │
└────────┴───────────────┴────────────┴──────────────┘

s3ClusterpartClickHouseCloud3ClickHouse

┌─n─┬─parts─┬─rows_total───┐
│ 1 │ 86.00 │ 1.61 billion │
│ 2 │ 76.00 │ 1.52 billion │
│ 3 │ 78.00 │ 1.51 billion │
└───┴───────┴──────────────┘

part

ClickHouse1706part

┌─merges─┐
│   1706 │
└────────┘

Keeper

ClickHouseKeeperClickHousepartpartpartClickHouse使KeeperpartpartpartblobKeeper

partpart18,000Keeper12,000ClickHouseKeeper800

total_requests:      17705
multi_requests:      11642
watch_notifications: 822

ClickHouse

┌─n─┬─total_requests─┬─multi_requests─┬─watch_notifications─┐
│ 1 │           5741 │           3671 │                 278 │
│ 2 │           5593 │           3685 │                 269 │
│ 3 │           6371 │           4286 │                 275 │
└───┴────────────────┴────────────────┴─────────────────────┘

Keeper

70%Keeper

KeeperClickHouseKeeper

ClickHouse

使1033使SharedMergeTree

0 rows in set. Elapsed: 33.634 sec. Processed 4.64 billion rows, 40.58 GB (138.01 million rows/s., 1.21 GB/s.)
Peak memory usage: 57.09 GiB.

3Keeper

total_requests:      60925
multi_requests:      41767
watch_notifications: 3468

使3ClickHousepart~25使使~1partClickHouse使9

0 rows in set. Elapsed: 121.421 sec. Processed 4.64 billion rows, 40.58 GB (38.23 million rows/s., 334.19 MB/s.)
Peak memory usage: 12.02 GiB.

part2404

┌─parts─────────┬─rows_avg─────┬─size_avg─┬─rows_total───┐
│ 4.24 thousand │ 1.09 million │ 9.20 MiB │ 4.64 billion │
└───────────────┴──────────────┴──────────┴──────────────┘

part

┌─merges─┐
│   9094 │
└────────

Keeper147k17k

total_requests:      147540
multi_requests:      105951
watch_notifications: 7439

使~1partWikiStat24kpart

─parts──────────┬─rows_avg─────┬─size_avg─┬─rows_total────┐
│ 23.75 thousand │ 1.10 million │ 9.24 MiB │ 26.23 billion │
└────────────────┴──────────────┴──────────┴───────────────┘

┌─merges─┐
│  28959 │
└────────┘

680kKeeper

total_requests:      680996
multi_requests:      474093
watch_notifications: 32779


Keeper

`keeper-bench-suite`ClickHouseKeeper`keeper-bench-suite`N3ClickHouseKeeper

`keeper-bench`KeeperZooKeeperNClickHouseKeeper`KeeperBenchSuite`使Keeper

使AWSEC2Python

3EC2m6a.4xlargeKeeperDockercAdvisorRediscAdvisor3Keeper

`keeper-bench`使

cAdvisorKeeperPrometheus

ClickHouseCloudSQLGrafana便

ClickHouseKeeperZooKeeperPrometheusCPU使cAdvisorKeeperDocker使KeeperCPU


Keeper

使DockerClickHouseKeeperZooKeeper1CPU+1GBRAM3CPU+1GBRAM6CPU+6GBRAM

Keeper使`keeper-bench`ClickHouseKeeper3101005001000

Keeper使`keeper-bench`Keeper1~1000使

ClickHouse1/32/3


Prometheus

使cAdvisorPrometheus

  • 使container_memory_working_set_bytes
  • CPU使container_cpu_usage_seconds_total

使ClickHouseKeeperZooKeeperPrometheusKeeperPrometheusZooKeeperJVM使

Keeper


使`keeper-bench-suite`ClickHouseKeeperZooKeeper10ClickHouseCloud使SQL


  • 95
  • 99

使ClickHouseKeeper23.5ZooKeeper3.8OpenJDK11216


3ClickHouseKeeper使3CPU2GBRAM99

使

ClickHouseKeeperZooKeeper使3ClickHouse640ClickHouseKeeper使ZooKeeper46

ZooKeeper使1GiBJVMJVMFLAGS-Xmx1024m-Xms1024mJVMJava使~1GiB使Docker使JVM使JVMJVM使JVMClickHouseKeeper使

ZooKeeper使1GiBJVM使2GiBJVMZooKeeper使2GiBJVM1.56GiBClickHouseKeeperZooKeeper

CPU使

CPU使

ClickHouseKeeperZooKeeperClickHouseKeeper使CPU使


Keeper

ClickHouseCloudKeeper使ClickHouseCloudKeeperClickHouse使Keeper


-2-2part使KeeperKeeperpartpartblobKeeperznodepartpart使ClickHousepartznodesClickHouseKeeperKeeper


part

server-2partpart使Keeperpart-2partpart使Keeperpart

KeeperKeeperClickHousepartClickHouseKeeper线


线

ZooKeeperClickHouseKeeperZABRaftpart

ZABZooKeeper2008

Raft2021KeeperC++

线使ClickHouseKeeperznodesKeeper线

ZooKeeper使线KeeperNuRaft使线

线使CPUZooKeeperClickHouseKeeper99

ZooKeeperClickHouseKeeper使136CPU500128

线ZABRaftCPU


KeeperKeeperRaft

KeeperKeeperRaft

线Multi-groupRaftKeeper使RaftKeeper

RaftKeeper/ClickHouse

ClickHouseKeeperZooKeeperClickHouseCloud使ClickHouseKeeper使线


云数据库 ClickHouse 版是阿里云提供的全托管 ClickHouse服务,是国内唯一和 ClickHouse 原厂达成战略合作并一方提供企业版内核服务的云产品。 企业版较社区版 ClickHouse 增强支持实时update&delete,云原生存算分离及Serverless 能力,整体成本可降低50%以上,现已开启邀测,欢迎申请体验(链接:https://www.aliyun.com/product/apsaradb/clickhouse

产品介绍(https://www.aliyun.com/product/apsaradb/clickhouse

技术交流群:

image.png

ClickHouse官方公众号:

image.png

相关文章
|
存储 分布式计算 数据挖掘
clickhouse集群zookeeper平滑搬迁实践
clickhouse集群zookeeper平滑搬迁实践
714 0
clickhouse集群zookeeper平滑搬迁实践
|
存储 网络协议 Cloud Native
ClickHouse Keeper 源码解析
ClickHouse 社区在21.8版本中引入了 ClickHouse Keeper。ClickHouse Keeper 是完全兼容 Zookeeper 协议的分布式协调服务。本文对开源版本 ClickHouse v21.8.10.19-lts 源码进行了解析。
ClickHouse Keeper 源码解析
|
存储 索引
67.【clickhouse】ClickHouse从入门到放弃-对于分区、索引、标记和压缩数据的协同总结
【clickhouse】ClickHouse从入门到放弃-对于分区、索引、标记和压缩数据的协同总结
67.【clickhouse】ClickHouse从入门到放弃-对于分区、索引、标记和压缩数据的协同总结
|
存储 缓存 数据库
66.【clickhouse】ClickHouse从入门到放弃-数据标记
【clickhouse】ClickHouse从入门到放弃-数据标记
66.【clickhouse】ClickHouse从入门到放弃-数据标记
|
存储 算法 NoSQL
65.【clickhouse】ClickHouse从入门到放弃-数据存储
【clickhouse】ClickHouse从入门到放弃-数据存储
65.【clickhouse】ClickHouse从入门到放弃-数据存储
|
存储 数据库 索引
64.【clickhouse】ClickHouse从入门到放弃-二级索引
【clickhouse】ClickHouse从入门到放弃-二级索引
64.【clickhouse】ClickHouse从入门到放弃-二级索引
|
存储 算法 数据库
63.【clickhouse】ClickHouse从入门到放弃-一级索引
【clickhouse】ClickHouse从入门到放弃-一级索引
63.【clickhouse】ClickHouse从入门到放弃-一级索引
|
存储 算法 数据库
62.【clickhouse】ClickHouse从入门到放弃-数据分区
【clickhouse】ClickHouse从入门到放弃-数据分区
62.【clickhouse】ClickHouse从入门到放弃-数据分区
|
存储 数据库 索引
61.【clickhouse】ClickHouse从入门到放弃-MergeTree的存储结构
【clickhouse】ClickHouse从入门到放弃-MergeTree的存储结构
61.【clickhouse】ClickHouse从入门到放弃-MergeTree的存储结构

相关产品

  • 云原生数据仓库AnalyticDB MySQL版
  • 云原生数据仓库 AnalyticDB PostgreSQL版