mysql批量更新时,canal-kafka消息重复,且似乎进入了死循环?

canal 版:canal-1.1.3 mysql版本:'5.7.20 (windows运行) kakfa:kafka_2.11-0.10.2.1 zk:3.4.8

canal 配置:

主配置:https://raw.githubusercontent.com/myleolei/canal-issues-config/master/canal%20%E4%B8%BB%E9%85%8D%E7%BD%AE.txt

instance配置:https://raw.githubusercontent.com/myleolei/canal-issues-config/master/instance%E9%85%8D%E7%BD%AE.txt

mysql create table ddl: CREATE TABLE spider_store ( id bigint(20) NOT NULL AUTO_INCREMENT, src_url varchar(2000) DEFAULT NULL, cn_text longtext, html longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci, PRIMARY KEY (id) ) ENGINE=InnoDB AUTO_INCREMENT=3076 DEFAULT CHARSET=utf8

重现步骤:

往表插入约2000条数据,cn_text,html 的内容都比较长,约1万字符,html字段包含中文,emoji表情

执行:update chafujin.spider_store set cn_text=CONCAT(cn_text,'aaaaaa');

启动canal(启动前指定bin起始位置为update语句之前)

表现:

kafka不停输出消息,消息似乎在循环输出,batchId也在增长,累计输出6千+,直到canal关闭 查看日志,大量的:

2019-08-29 19:12:51.080 [pool-8-thread-2] ERROR com.alibaba.otter.canal.server.CanalMQStarter - rollback error, clientId:1001 batchId:28281 is not exist , please check com.alibaba.otter.canal.server.exception.CanalServerException: rollback error, clientId:1001 batchId:28281 is not exist , please check 2019-08-29 19:12:51.080 [pool-8-thread-2] ERROR com.alibaba.otter.canal.server.CanalMQStarter - rollback error, clientId:1001 batchId:28281 is not exist , please check com.alibaba.otter.canal.server.exception.CanalServerException: rollback error, clientId:1001 batchId:28281 is not exist , please check 2019-08-29 19:12:51.181 [pool-8-thread-2] ERROR com.alibaba.otter.canal.kafka.CanalKafkaProducer - no match ack positionLogPosition[identity=LogIdentity[sourceAddress=/192.168.56.1:3306,slaveId=-1],postion=EntryPosition[included=false,journalName=mysql-bin.000001,position=186426,serverId=1,gtid=,timestamp=1567064958000]] com.alibaba.otter.canal.store.CanalStoreException: no match ack positionLogPosition[identity=LogIdentity[sourceAddress=/192.168.56.1:3306,slaveId=-1],postion=EntryPosition[included=false,journalName=mysql-bin.000001,position=186426,serverId=1,gtid=,timestamp=1567064958000]]

原提问者GitHub用户myleolei