AI 助理
文档备案控制台
开发者社区 云计算 文章 正文

Apache ZooKeeper - 构建ZooKeeper源码环境及StandAlone模式下的服务端和客户端启动

2024-01-02 335
版权
版权声明：
本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《 阿里云开发者社区用户服务协议》和 《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写 侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。
简介： Apache ZooKeeper - 构建ZooKeeper源码环境及StandAlone模式下的服务端和客户端启动

源码环境搭建

有一天大佬问我看过zk的源码吗…？

好的好的 谁说不看呢

说干就干

clone source

https://github.com/apache/zookeeper

当然了，我要看 3.5.8这个版本的

https://github.com/apache/zookeeper/tree/branch-3.5.8

Clone ---- HTTPS

IDEA里

源码结构

启动

查找启动类

根据经验，从启动脚本中去找启动类 ，八成启动类中有个main方法

找启动脚本的 start方法

找到 ZOOMAIN="org.apache.zookeeper.server.quorum.QuorumPeerMain"

ZOOCFG="$2" 可知是通过外部传参进来的，那就按照约定的来吧

配置文件

从 zoo_sample.cfg copy一个出来

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=D:\IdeaProjects\zookeeperSource\data\zk
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

pom文件调整

将zookeeper-server项目里pom.xml文件里依赖的包的scope为provided这一行全部注释掉 【jline除外】

log4j.properties

将conf文件夹里的log4j.properties文件复制一份到zookeeper-server项目的 \target\classes 目录下,不然没有日志

以源码的方式启动Server

2020-11-27 11:13:47,849 [myid:] - INFO  [main:QuorumPeerConfig@135] - Reading configuration from: D:\IdeaProjects\zookeeperSource\conf\zoo.cfg
2020-11-27 11:13:47,858 [myid:] - WARN  [main:VerifyingFileFactory@59] - D:IdeaProjectszookeeperSourcedata is relative. Prepend .\ to indicate that you're sure!
2020-11-27 11:13:47,870 [myid:] - INFO  [main:QuorumPeerConfig@387] - clientPortAddress is 0.0.0.0:2181
2020-11-27 11:13:47,872 [myid:] - INFO  [main:QuorumPeerConfig@391] - secureClientPort is not set
2020-11-27 11:13:47,876 [myid:] - INFO  [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2020-11-27 11:13:47,876 [myid:] - INFO  [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2020-11-27 11:13:47,876 [myid:] - INFO  [main:DatadirCleanupManager@101] - Purge task is not scheduled.
2020-11-27 11:13:47,876 [myid:] - WARN  [main:QuorumPeerMain@125] - Either no config or no quorum defined in config, running  in standalone mode
2020-11-27 11:13:47,882 [myid:] - INFO  [main:ManagedUtil@45] - Log4j 1.2 jmx support found and enabled.
2020-11-27 11:13:47,951 [myid:] - INFO  [main:QuorumPeerConfig@135] - Reading configuration from: D:\IdeaProjects\zookeeperSource\conf\zoo.cfg
2020-11-27 11:13:47,951 [myid:] - WARN  [main:VerifyingFileFactory@59] - D:IdeaProjectszookeeperSourcedata is relative. Prepend .\ to indicate that you're sure!
2020-11-27 11:13:47,952 [myid:] - INFO  [main:QuorumPeerConfig@387] - clientPortAddress is 0.0.0.0:2181
2020-11-27 11:13:47,952 [myid:] - INFO  [main:QuorumPeerConfig@391] - secureClientPort is not set
2020-11-27 11:13:47,952 [myid:] - INFO  [main:ZooKeeperServerMain@117] - Starting server
2020-11-27 11:13:47,955 [myid:] - INFO  [main:FileTxnSnapLog@115] - zookeeper.snapshot.trust.empty : false
2020-11-27 11:13:47,985 [myid:] - INFO  [main:Environment@109] - Server environment:zookeeper.version=1.0.0-1, built on 2020-11-27
2020-11-27 11:13:47,987 [myid:] - INFO  [main:Environment@109] - Server environment:host.name=LAPTOP-JF3RBRRJ
2020-11-27 11:13:47,987 [myid:] - INFO  [main:Environment@109] - Server environment:java.version=1.8.0_261
2020-11-27 11:13:47,987 [myid:] - INFO  [main:Environment@109] - Server environment:java.vendor=Oracle Corporation
2020-11-27 11:13:47,987 [myid:] - INFO  [main:Environment@109] - Server environment:java.home=D:\Program Files (x86)\Java\jdk1.8.0_261\jre
2020-11-27 11:13:47,987 [myid:] - INFO  [main:Environment@109] - Server environment:java.class.path=D:\Program Files
......
......
......
2020-11-27 11:13:47,987 [myid:] - INFO  [main:Environment@109] - Server environment:java.io.tmpdir=C:\Users\artisan\AppData\Local\Temp\
2020-11-27 11:13:47,987 [myid:] - INFO  [main:Environment@109] - Server environment:java.compiler=<NA>
2020-11-27 11:13:47,987 [myid:] - INFO  [main:Environment@109] - Server environment:os.name=Windows 10
2020-11-27 11:13:47,988 [myid:] - INFO  [main:Environment@109] - Server environment:os.arch=amd64
2020-11-27 11:13:47,988 [myid:] - INFO  [main:Environment@109] - Server environment:os.version=10.0
2020-11-27 11:13:47,988 [myid:] - INFO  [main:Environment@109] - Server environment:user.name=artisan
2020-11-27 11:13:47,988 [myid:] - INFO  [main:Environment@109] - Server environment:user.home=C:\Users\artisan
2020-11-27 11:13:47,988 [myid:] - INFO  [main:Environment@109] - Server environment:user.dir=D:\IdeaProjects\zookeeperSource
2020-11-27 11:13:47,988 [myid:] - INFO  [main:Environment@109] - Server environment:os.memory.free=222MB
2020-11-27 11:13:47,988 [myid:] - INFO  [main:Environment@109] - Server environment:os.memory.max=3499MB
2020-11-27 11:13:47,988 [myid:] - INFO  [main:Environment@109] - Server environment:os.memory.total=236MB
2020-11-27 11:13:47,990 [myid:] - INFO  [main:ZooKeeperServer@938] - minSessionTimeout set to 4000
2020-11-27 11:13:47,990 [myid:] - INFO  [main:ZooKeeperServer@947] - maxSessionTimeout set to 40000
2020-11-27 11:13:47,991 [myid:] - INFO  [main:ZooKeeperServer@166] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir D:IdeaProjectszookeeperSourcedata\version-2 snapdir D:IdeaProjectszookeeperSourcedata\version-2
2020-11-27 11:13:48,014 [myid:] - INFO  [main:Log@169] - Logging initialized @433ms to org.eclipse.jetty.util.log.Slf4jLog
2020-11-27 11:13:48,167 [myid:] - WARN  [main:ContextHandler@1520] - o.e.j.s.ServletContextHandler@215be6bb{/,null,UNAVAILABLE} contextPath ends with /*
2020-11-27 11:13:48,167 [myid:] - WARN  [main:ContextHandler@1531] - Empty contextPath
2020-11-27 11:13:48,182 [myid:] - INFO  [main:Server@359] - jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 1.8.0_261-b12
2020-11-27 11:13:48,223 [myid:] - INFO  [main:DefaultSessionIdManager@333] - DefaultSessionIdManager workerName=node0
2020-11-27 11:13:48,223 [myid:] - INFO  [main:DefaultSessionIdManager@338] - No SessionScavenger set, using defaults
2020-11-27 11:13:48,226 [myid:] - INFO  [main:HouseKeeper@140] - node0 Scavenging every 660000ms
2020-11-27 11:13:48,235 [myid:] - INFO  [main:ContextHandler@825] - Started o.e.j.s.ServletContextHandler@215be6bb{/,null,AVAILABLE}
2020-11-27 11:13:48,462 [myid:] - INFO  [main:AbstractConnector@330] - Started ServerConnector@17f6480{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2020-11-27 11:13:48,463 [myid:] - INFO  [main:Server@399] - Started @882ms
2020-11-27 11:13:48,463 [myid:] - INFO  [main:JettyAdminServer@112] - Started AdminServer on address 0.0.0.0, port 8080 and command URL /commands
2020-11-27 11:13:48,468 [myid:] - INFO  [main:ServerCnxnFactory@135] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
2020-11-27 11:13:48,475 [myid:] - INFO  [main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s sessionless connection timeout, 2 selector thread(s), 24 worker threads, and 64 kB direct buffers.
2020-11-27 11:13:48,481 [myid:] - INFO  [main:NIOServerCnxnFactory@686] - binding to port 0.0.0.0/0.0.0.0:2181
2020-11-27 11:13:48,498 [myid:] - INFO  [main:ZKDatabase@117] - zookeeper.snapshotSizeFactor = 0.33
2020-11-27 11:13:48,511 [myid:] - INFO  [main:FileSnap@83] - Reading snapshot D:IdeaProjectszookeeperSourcedata\version-2\snapshot.0
2020-11-27 11:13:48,527 [myid:] - INFO  [main:FileTxnSnapLog@404] - Snapshotting: 0x4 to D:IdeaProjectszookeeperSourcedata\version-2\snapshot.4
2020-11-27 11:13:48,547 [myid:] - INFO  [main:ContainerManager@64] - Using checkIntervalMs=60000 maxPerMinute=10000

看重点

2020-11-27 11:13:48,463 [myid:] - INFO [main:Server@399] - Started @882ms

2020-11-27 11:13:48,481 [myid:] - INFO [main:NIOServerCnxnFactory@686] - binding to port 0.0.0.0/0.0.0.0:2181

以源码的方式启动Client

启动日志

Connecting to 127.0.0.1:2181
2020-11-27 11:05:41,288 [myid:] - INFO  [main:Environment@109] - Client environment:zookeeper.version=1.0.0-1, built on 2020-11-27
2020-11-27 11:05:41,291 [myid:] - INFO  [main:Environment@109] - Client environment:host.name=LAPTOP-JF3RBRRJ
2020-11-27 11:05:41,291 [myid:] - INFO  [main:Environment@109] - Client environment:java.version=1.8.0_261
2020-11-27 11:05:41,292 [myid:] - INFO  [main:Environment@109] - Client environment:java.vendor=Oracle Corporation
2020-11-27 11:05:41,292 [myid:] - INFO  [main:Environment@109] - Client environment:java.home=D:\Program Files (x86)\Java\jdk1.8.0_261\jre
2020-11-27 11:05:41,292 [myid:] - INFO  [main:Environment@109] - Client environment:java.class.path=D:\Program Files  
..........
..........
..........
2020-11-27 11:05:41,293 [myid:] - INFO  [main:Environment@109] - Client environment:java.io.tmpdir=C:\Users\artisan\AppData\Local\Temp\
2020-11-27 11:05:41,293 [myid:] - INFO  [main:Environment@109] - Client environment:java.compiler=<NA>
2020-11-27 11:05:41,293 [myid:] - INFO  [main:Environment@109] - Client environment:os.name=Windows 10
2020-11-27 11:05:41,293 [myid:] - INFO  [main:Environment@109] - Client environment:os.arch=amd64
2020-11-27 11:05:41,293 [myid:] - INFO  [main:Environment@109] - Client environment:os.version=10.0
2020-11-27 11:05:41,293 [myid:] - INFO  [main:Environment@109] - Client environment:user.name=artisan
2020-11-27 11:05:41,293 [myid:] - INFO  [main:Environment@109] - Client environment:user.home=C:\Users\artisan
2020-11-27 11:05:41,293 [myid:] - INFO  [main:Environment@109] - Client environment:user.dir=D:\IdeaProjects\zookeeperSource
2020-11-27 11:05:41,293 [myid:] - INFO  [main:Environment@109] - Client environment:os.memory.free=224MB
2020-11-27 11:05:41,294 [myid:] - INFO  [main:Environment@109] - Client environment:os.memory.max=3499MB
2020-11-27 11:05:41,294 [myid:] - INFO  [main:Environment@109] - Client environment:os.memory.total=236MB
2020-11-27 11:05:41,296 [myid:] - INFO  [main:ZooKeeper@868] - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@22d8cfe0
2020-11-27 11:05:41,301 [myid:] - INFO  [main:X509Util@79] - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2020-11-27 11:05:41,521 [myid:] - INFO  [main:ClientCnxnSocket@237] - jute.maxbuffer value is 4194304 Bytes
2020-11-27 11:05:41,527 [myid:] - INFO  [main:ClientCnxn@1653] - zookeeper.request.timeout value is 0. feature enabled=
Welcome to ZooKeeper!
JLine support is disabled
2020-11-27 11:05:41,539 [myid:127.0.0.1:2181] - INFO  [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@1112] - Opening socket connection to server kubernetes.docker.internal/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2020-11-27 11:05:41,540 [myid:127.0.0.1:2181] - INFO  [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@959] - Socket connection established, initiating session, client: /127.0.0.1:50568, server: kubernetes.docker.internal/127.0.0.1:2181
2020-11-27 11:05:41,562 [myid:127.0.0.1:2181] - INFO  [main-SendThread(127.0.0.1:2181):ClientCnxn$SendThread@1394] - Session establishment complete on server kubernetes.docker.internal/127.0.0.1:2181, sessionid = 0x1004701bbe50000, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null

验证

在客户端执行ZK命令

好了基本完事儿

遇到的问题

缺少Info接口

Info是用来发布的时候生成版本用的，正规的话，需要用 org.apache.zookeeper.version.util.VerGen生成Info ， 我们这里随便搞个就行了

package org.apache.zookeeper.version;
/**
 * @author 小工匠
 * @version 1.0
 * @description: TODO
 * @date 2020/11/27 9:54
 * @mark: show me the code , change the world
 */
public interface Info {
    int MAJOR = 1;
    int MINOR = 0;
    int MICRO = 0 ;
    String QUALIFIER = null;
    int REVISION = -1;
    String REVISION_HASH = "1";
    String BUILD_DATE = "2020-11-27";
}

缺少org.apache.zookeeper.data.ACL

重新编译 Jute序列化模块

Could not find artifact pl.project13.maven

Could not find artifact pl.project13.maven:git-commit-id-plugin:jar: in aliyunmaven (https://maven.aliyun.com/repository/central)

忽略即可


文章标签：
微服务引擎
Apache
关键词：
微服务引擎环境
Apache构建
Apache服务端
Apache客户端
Apache模式
小小工匠
目录
相关文章
Deephub
|
7月前
|
数据采集 机器学习/深度学习 数据可视化
让回归模型不再被异常值"带跑偏"，MSE和Cauchy损失函数在噪声数据环境下的实战对比
本文探讨了MSE与Cauchy损失函数在线性回归中的表现，特别是在含噪声数据环境下的差异。研究发现，MSE虽具良好数学性质，但对异常值敏感；而Cauchy通过其对数惩罚机制降低异常值影响，展现出更强稳定性。实验结果表明，Cauchy损失函数在处理含噪声数据时参数估计更接近真实值，为实际应用提供了更鲁棒的选择。
Deephub
295 1
让回归模型不再被异常值"带跑偏"，MSE和Cauchy损失函数在噪声数据环境下的实战对比
蜗牛ICU
|
存储 API Apache
【zookeeper 第三篇章】客户端 API
本文介绍了Apache ZooKeeper客户端的一些常用命令及其用法。首先，`create`命令用于创建不同类型的节点并为其赋值，如持久化节点、有序节点及临时节点等。通过示例展示了如何创建这些节点，并演示了创建过程中的输出结果。其次，`ls`命令用于列出指定路径下的所有子节点。接着，`set`命令用于更新节点中的数据，可以指定版本号实现乐观锁机制。
蜗牛ICU
257 0
阿里云云原生
|
存储 运维 安全
防盗、防泄露、防篡改，我们把 ZooKeeper 的这种认证模式玩明白了
ZooKeeper 作为应用的核心中间件在业务流程中存储着敏感数据，具有关键作用。正确且规范的使用方法对确保数据安全至关重要，否则可能会因操作不当而导致内部数据泄露，进而带来严重的安全风险。因此，在日常的 ZooKeeper 运维和使用过程中，标准化和安全的操作对于加强企业安全防护和能力建设显得格外关键。为了实现这一目标，MSE 提供了一整套标准化流程，帮助用户以更安全、更简便的方式使用 ZooKeeper，从而加速企业安全能力的提升同时最大程度地降低在变更过程中可能出现的风险。
阿里云云原生
9461 102
武子康
|
分布式计算 Java Hadoop
Hadoop-30 ZooKeeper集群 JavaAPI 客户端 POM Java操作ZK 监听节点 监听数据变化 创建节点 删除节点
Hadoop-30 ZooKeeper集群 JavaAPI 客户端 POM Java操作ZK 监听节点 监听数据变化 创建节点 删除节点
武子康
270 1
1176112968452250
|
负载均衡 API 数据安全/隐私保护
Zookeeper的客户端-原生的API
Zookeeper的客户端-原生的API
1176112968452250
151 1
不吃核桃
|
数据采集 分布式计算 Kubernetes
Apache Flink 实践问题之ZooKeeper 网络瞬断时如何解决
Apache Flink 实践问题之ZooKeeper 网络瞬断时如何解决
不吃核桃
371 4
武子康
|
存储 SQL 消息中间件
Hadoop-26 ZooKeeper集群 3台云服务器 基础概念简介与环境的配置使用 架构组成 分布式协调框架 Leader Follower Observer
Hadoop-26 ZooKeeper集群 3台云服务器 基础概念简介与环境的配置使用 架构组成 分布式协调框架 Leader Follower Observer
武子康
235 0
JavaSouth南哥
|
API
【想进大厂还不会阅读源码】ShenYu源码-替换ZooKeeper客户端
ShenYu源码阅读。相信大家碰到源码时经常无从下手，不知道从哪开始阅读😭。我认为有一种办法可以解决大家的困扰！至此，我们发现自己开始从大量堆砌的源码中脱离开来😀。ShenYu是一个异步的，高性能的，跨语言的，响应式的 API 网关。
JavaSouth南哥
211 3
wljslmz
|
分布式计算 监控 Hadoop
详解 Apache ZooKeeper 和 Apache Oozie
【8月更文挑战第31天】
wljslmz
423 0
yuanzhengme
|
Java API Apache
ZooKeeper【基础 03】Java 客户端 Apache Curator 基础 API 使用举例（含源代码）
【4月更文挑战第11天】ZooKeeper【基础 03】Java 客户端 Apache Curator 基础 API 使用举例（含源代码）
yuanzhengme
219 11

热门文章

最新文章

  • 1
    zookeeper集群搭建
  • 2
    Spring Cloud Zookeeper 中心化配置文件
  • 3
    防盗、防泄露、防篡改，我们把 ZooKeeper 的这种认证模式玩明白了
  • 4
    阿里云微服务引擎及 API 网关 2024 年 5 月产品动态
  • 5
    【ZooKeeper Notes 24】Leader选举杂记1
  • 6
    Zookeeper面试题总结
  • 7
    无须托管应用就能享有微服务治理 | 微服务引擎 MSE 重磅升级
  • 8
    [Dubbo日记]（1）Zookeeper安装
  • 9
    十、Zookeeper （leader）选举机制
  • 10
    搜索服务Solr集群搭建 使用ZooKeeper作为代理层
  • 1
    2025 OSCAR丨与创新者同频！Apache RocketMQ 邀您共赴开源之约
    190
  • 2
    Confluent 首席架构师万字剖析 Apache Fluss（三）：湖流一体
    570
  • 3
    Confluent 首席架构师万字剖析 Apache Fluss（二）：核心架构
    483
  • 4
    Apache ShenYu 架构学习指南
    603
  • 5
    阿里云、Ververica、Confluent 与 LinkedIn 携手推进流式创新，共筑基于 Apache Flink Agents 的智能体 AI 未来
    674
  • 6
    Apache Doris 与 ClickHouse：运维与开源闭源对比
    563
  • 7
    Confluent 首席架构师万字剖析 Apache Fluss（一）：核心概念
    645
  • 8
    Apache Doris 4.0 AI 能力揭秘（二）：为企业级应用而生的 AI 函数设计与实践
    398
  • 9
    Apache Doris 3.1 正式发布：半结构化分析全面升级，湖仓一体能力再跃新高
    709
  • 10
    Apache Kafka 分布式流处理平台技术详解与实践指南
    472

    • 相关课程

    更多
  • 基于MSE的大促场景流量入口防护最佳实践
  • Dubbo + ZooKeeper 的服务发现最佳实践
  • MSE微服务测试最佳实践 - 自动化回归
  • 基于Zookeeper、Dubbo构建互联网分布式基础架构
  • 大数据ZooKeeper快速入门
  • 分布式协调系统 Zookeeper 快速入门

    • 相关电子书

    更多
  • 《MSE 微服务网关》
  • 微服务引擎 MSE 治理中心重磅发布
  • 阿里云微服务引擎 MSE 2.0 线上发布
    • 下一篇
    附部署代码｜云数据库RDS 全托管 Supabase服务：小白轻松搞定开发AI应用