开发者社区> 问答> 正文

mysql操作含大事物,canal获取binlog对象flatMessage含多条数据的变化前后结果

1、在mysql执行update table set fieldX = ''; row changes: 8000*

2、CanalServerWithEmbedded中getWithoutAck获取binlog结果日志:

getWithoutAck successfully, raw:false, clientId:1001 batchSize:16 real size is 1 and result is [batchId:2764 , position:PositionRange[start=LogPosition[identity=LogIdentity[sourceAddress=,slaveId=-1],postion=EntryPosition[included=false,journalName=mysql-bin.000194,position=47554093,serverId=150593306,gtid=,timestamp=1591318942000]],ack=,end=LogPosition[identity=LogIdentity[sourceAddress=,slaveId=-1],postion=EntryPosition[included=false,journalName=mysql-bin.000194,position=47554093,serverId=150593306,gtid=,timestamp=1591318942000]],endSeq=2]]

3、推送MQ报文超出限制:

c.a.o.c.r.CanalRocketMQProducer [186][pool-4-thread-1] - send flat message to fixed partition error, messageSize: 54999.46KB, [table: ***, sql: ] org.apache.rocketmq.client.exception.MQClientException: CODE: 13 DESC: the message body size over max value, MAX: 52428800 For more information, please visit the url, http://rocketmq.apache.org/docs/faq/ at org.apache.rocketmq.client.Validators.checkMessage(Validators.java:99) ~[rocketmq-client-4.5.2.jar:4.5.2] at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendSelectImpl(DefaultMQProducerImpl.java:1080) ~[rocketmq-client-4.5.2.jar:4.5.2] at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1068) ~[rocketmq-client-4.5.2.jar:4.5.2] at org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl.send(DefaultMQProducerImpl.java:1063) ~[rocketmq-client-4.5.2.jar:4.5.2] at org.apache.rocketmq.client.producer.DefaultMQProducer.send(DefaultMQProducer.java:524) ~[rocketmq-client-4.5.2.jar:4.5.2] at com.alibaba.otter.canal.rocketmq.CanalRocketMQProducer.sendMessage(CanalRocketMQProducer.java:201) [canal.server-1.1.4.jar:na] at com.alibaba.otter.canal.rocketmq.CanalRocketMQProducer.send(CanalRocketMQProducer.java:184) [canal.server-1.1.4.jar:na] at com.alibaba.otter.canal.rocketmq.CanalRocketMQProducer.send(CanalRocketMQProducer.java:89) [canal.server-1.1.4.jar:na] at com.alibaba.otter.canal.server.CanalMQStarter.worker(CanalMQStarter.java:183) [canal.server-1.1.4.jar:na] at com.alibaba.otter.canal.server.CanalMQStarter.access$500(CanalMQStarter.java:23) [canal.server-1.1.4.jar:na] at com.alibaba.otter.canal.server.CanalMQStarter$CanalMQRunnable.run(CanalMQStarter.java:225) [canal.server-1.1.4.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_232] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_232] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_232]

4、发送MQ异常,触发canalServer.rollback。导致读取MQ一直卡在一个位置点,无法继续执行

原提问者GitHub用户thelifeskygit

展开
收起
Java工程师 2023-05-03 10:02:10 75 0
1 条回答
写回答
取消 提交回答
  • 控制一下batch参数来减少发送包的大小

    原回答者GitHub用户agapple

    2023-05-04 13:40:38
    赞同 展开评论 打赏
问答排行榜
最热
最新

相关电子书

更多
搭建电商项目架构连接MySQL 立即下载
搭建4层电商项目架构,实战连接MySQL 立即下载
PolarDB-X 2.0 全局 Binlog 与备份恢复能 立即下载

相关镜像