从MariaDB同步数据到Kafka
表:
CREATE TABLE temp_20220615_1 ( ID int(11) NOT NULL COMMENT 'ID', addDate timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP() COMMENT '添加时间', updateDate timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP() ON UPDATE CURRENT_TIMESTAMP() COMMENT '更新时间', PRIMARY KEY (ID) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; CREATE TABLE temp_20220615_2 ( ID int(11) NOT NULL COMMENT 'ID', addDate timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) COMMENT '添加时间', updateDate timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3) COMMENT '更新时间', PRIMARY KEY (ID) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; CREATE TABLE temp_20220615_3 ( ID int(11) NOT NULL COMMENT 'ID', addDate timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) COMMENT '添加时间', updateDate timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6) COMMENT '更新时间', PRIMARY KEY (ID) ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
timestamp类型,如果为秒级别的,可以正常解析和消费,如果为timestamp(3)毫秒级别或者 timestamp(6)微秒级别,就会报错
2022-06-15 07:36:34.808 [destination = xxx , address = /x.x.x.x:3306 , EventParser] ERROR com.alibaba.otter.canal.common.alarm.LogAlarmHandler - destination:col_album[com.alibaba.otter.canal.parse.exception.CanalParseException: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed. Caused by: com.alibaba.otter.canal.parse.exception.CanalParseException: parse row data failed. Caused by: java.lang.IllegalArgumentException: limit excceed: 21 at com.taobao.tddl.dbsync.binlog.LogBuffer.getUint32(LogBuffer.java:562) at com.taobao.tddl.dbsync.binlog.event.RowsLogBuffer.fetchValue(RowsLogBuffer.java:438) at com.taobao.tddl.dbsync.binlog.event.RowsLogBuffer.nextValue(RowsLogBuffer.java:125) at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.parseOneRow(LogEventConvert.java:752) at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.parseRowsEvent(LogEventConvert.java:566) at com.alibaba.otter.canal.parse.inbound.mysql.MysqlMultiStageCoprocessor$DmlParserStage.onEvent(MysqlMultiStageCoprocessor.java:341) at com.alibaba.otter.canal.parse.inbound.mysql.MysqlMultiStageCoprocessor$DmlParserStage.onEvent(MysqlMultiStageCoprocessor.java:327) at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
原提问者GitHub用户RankRao
升级下最新的1.1.7,修复过mysql和mariadb新版本的binlog解析串位问题
原回答者GitHub用户agapple
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。