使用canal 1.0.24版本, mysql版本为5.7.20版本 canal启动后,有如下日志:
WARN c.a.otter.canal.parse.inbound.mysql.MysqlEventParser - prepare to find start position just show master status 就不再有其他日志了,并且zookeeper下1001节点中没有cursor节点。
打开日志debug,一直输出日志如下: getWithoutAck successfully, clientId:1001 batchSize:1000 but result is null
另外,mysql开启了gtid_mode = ON, 在canal部署的机器上采用配置的账号连接数据库实例,执行show master status得到: +------------------+-----------+--------------+------------------+-------------------------------------------------------------------------------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+-----------+--------------+------------------+-------------------------------------------------------------------------------------------+ | mysql-bin.000506 | 428663296 | | | f6af278c-cf73-11e7-86ed-801844ea454c:1-6, fb35b001-cf73-11e7-9c62-801844e9f00c:1-87011966 | +------------------+-----------+--------------+------------------+-------------------------------------------------------------------------------------------+
因此我判断账号配置没有问题,应该是能获取得到binlog position的
之前权限设置为: GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON . TO 'canal'@'%'; 现在改为: RANT ALL PRIVILEGES ON . TO 'canal'@'%' ;
然后重新启动canal,这时出现如下日志: 2018-01-12 11:57:30.620 [destination = 10_64_1_198_3318_instance , address = /10.64.0.101:3318 , EventParser] INFO c.a.otter.canal.parse.inbound.mysql.MysqlConnection - COM_BINLOG_DUMP with position:BinlogDumpCommandPacket[binlogPosition=523583203,slaveServerId=1234,binlogFileName=mysql-bin.000506,command=18] 2018-01-12 11:57:30.633 [destination = 10_64_1_198_3318_instance , address = /10.64.0.101:3318 , EventParser] INFO com.taobao.tddl.dbsync.binlog.LogEvent - common_header_len= 19, number_of_event_types= 38 2018-01-12 11:57:32.987 [New I/O server worker #1-2] INFO c.a.otter.canal.server.embedded.CanalServerWithEmbedded - subscribe successfully, ClientIdentity[destination=10_64_1_198_3318_instance,clientId=1001,filter=] with first position:null 2018-01-12 11:57:33.002 [New I/O server worker #1-2] DEBUG c.a.otter.canal.server.embedded.CanalServerWithEmbedded - getWithoutAck successfully, clientId:1001 batchSize:1000 but result is null
貌似获取到了position,但是最后是null:...filter=] with first position:null
tcpdump抓包有 binlog的包进来, 看来是canal处理过程中的问题了,如果是binlog解析失败,应该会抛异常吧。 好奇怪
debug代码发现 canal.instance.memory.buffer.size = 1048576 canal.instance.memory.buffer.memunit = 1024
这两个属性相乘不能大于最大int,不然会溢出导致在计算当前buffer大小的时候出问题,无法put EVENT。 这块建议文档说明下,并且能够抛出异常就更好了
原提问者GitHub用户guorain321
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。