开发者社区> 问答> 正文

canal1.1.3使用dynamicTopic时ack失败导致zookeeper cursor不会

环境信息

canal version 1.1.3 mysql version 5.7.20

问题描述

使用Canal 1.1.3最新代码 ,启用dynamicTopic配置,使用zookeeper实现高可用 启动canal server一直报错:

2019-01-16 13:42:53.121 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:3 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:3 is not the firstly:1 2019-01-16 13:42:53.121 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:3 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:3 is not the firstly:1 2019-01-16 13:42:53.239 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:4 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:4 is not the firstly:1 2019-01-16 13:42:53.239 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:4 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:4 is not the firstly:1 2019-01-16 13:42:53.376 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:5 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:5 is not the firstly:1 2019-01-16 13:42:53.376 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:5 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:5 is not the firstly:1 2019-01-16 13:42:53.396 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:6 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:6 is not the firstly:1 2019-01-16 13:42:53.396 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:6 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:6 is not the firstly:1 2019-01-16 13:42:53.418 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:7 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:7 is not the firstly:1 2019-01-16 13:42:53.418 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:7 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:7 is not the firstly:1 2019-01-16 13:42:53.432 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:8 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:8 is not the firstly:1 2019-01-16 13:42:53.432 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:8 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:8 is not the firstly:1 2019-01-16 13:42:53.447 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:9 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:9 is not the firstly:1 2019-01-16 13:42:53.447 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:9 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:9 is not the firstly:1 2019-01-16 13:42:53.456 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:10 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:10 is not the firstly:1 2019-01-16 13:42:53.456 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:10 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:10 is not the firstly:1 2019-01-16 13:42:53.479 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:11 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:11 is not the firstly:1 2019-01-16 13:42:53.479 [pool-4-thread-1] ERROR com.alibaba.otter.canal.server.CanalMQStarter - batchId:11 is not the firstly:1 com.alibaba.otter.canal.meta.exception.CanalMetaManagerException: batchId:11 is not the firstly:1

ack失败导致zookeeper cursor不会更新

原提问者GitHub用户flackyang

展开
收起
古拉古拉 2023-05-08 14:59:38 99 0
2 条回答
写回答
取消 提交回答
  • 可以尝试一下1.1.3 alpha-3版本,ack的问题有修复

    原回答者GitHub用户agapple

    2023-05-09 18:06:53
    赞同 展开评论 打赏
  • 在Canal 1.1.3中,动态topic的Zookeeper实现需要配置正确,并且所连接的Zookeeper集群正常运行。从你提供的信息来看,Canal Server启动时存在错误,建议根据错误提示,尝试以下方法解决问题:

    1. 检查Zookeeper集群是否正常可用。尝试连接Zookeeper集群并且使用命令行进行数据操作,例如获取Zookeeper节点的信息。如果Zookeeper连接存在问题,需要检查Zookeeper配置、网络连接和Zookeeper集群状态等,并且确保Canal Server所连接的Zookeeper集群能够正常运行。

    2. 检查dynamicTopic的Zookeeper配置参数是否正确。在Canal Server配置文件中,需要指定Zookeeper集群所在的IP地址、端口号等信息,确保Canal Server能够正确连接到Zookeeper集群。同时,检查Zookeeper的命名空间、父节点等信息是否设置正确。

    3. 确认Canal Server中dynamicTopic的相关配置是否正确。dynamicTopic的相关配置连接了Zookeeper集群,根据指定的数据源配置动态创建binlog的topic,可以动态监听binlog,且动态监听到的binlog会写入对应动态创建的topic中。在Canal Server配置文件中,需要使用类似下面的配置对dynamicTopic进行设置:

    canal.instance.dynamicConfig=true
    canal.instance.zookeeper.address=127.0.0.1:2181
    canal.instance.zookeeper.sessionTimeout=6000
    canal.instance.zookeeper.connectTimeout= 6000
    canal.instance.zookeeper.namespace=example
    canal.instance.zookeeper.config.mode=full
    
    • canal.instance.dynamicConfig=true: 表示开启Canal Server的动态配置特性。
    • canal.instance.zookeeper.address: 表示Zookeeper的IP地址和端口号。
    • canal.instance.zookeeper.sessionTimeout: 表示Zookeeper的会话超时时间,单位为毫秒。
    • canal.instance.zookeeper.connectTimeout: 表示Zookeeper的连接超时时间,单位为毫秒。
    • canal.instance.zookeeper.namespace: 表示Zookeeper的命名空间。
    • canal.instance.zookeeper.config.mode: 表示Zookeeper的配置模式,使用full表示全量模式(每次启动配置都更新),使用lite表示简洁模式(只有当配置改变时才更新)。
    2023-05-08 15:17:17
    赞同 展开评论 打赏
问答排行榜
最热
最新

相关电子书

更多
《MSE 微服务网关》 立即下载
微服务引擎 MSE 治理中心重磅发布 立即下载
阿里云微服务引擎 MSE 2.0 线上发布 立即下载