使用canal时,发现当mysql库没有任何binlog产生时,关闭canal后部分mysql连接未释放。
mysql版本:5.6.24-72.2-log canal版本:1.0.21
关闭前的堆栈信息如下:
"destination = 90 , address = /192.168.12.110:3309 , HeartBeatTimeTask" daemon prio=10 tid=0x00007f4b2c00d000 nid=0x395b in Object.wait() [0x00007f4b51d13000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.util.TimerThread.mainLoop(Timer.java:552) - locked <0x000000076052ffa8> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:505)
"destination = 90 , address = /192.168.12.110:3309 , EventParser" prio=10 tid=0x00007f4bd073e800 nid=0x395a runnable [0x00007f4b51d54000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:197) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) - locked <0x0000000760a5efb0> (a java.lang.Object) at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.DirectLogFetcher.fetch0(DirectLogFetcher.java:154) at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.DirectLogFetcher.fetch(DirectLogFetcher.java:70) at com.alibaba.otter.canal.parse.inbound.mysql.MysqlConnection.dump(MysqlConnection.java:113) at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$3.run(AbstractEventParser.java:209) at java.lang.Thread.run(Thread.java:745)
关闭前的mysql连接信息如下:
mysql> show processlist; | 442033 | xxx | 192.168.201.174:47598 | NULL | Sleep | 4 | | NULL | 1 | 1 | | 442035 | xxx | 192.168.201.174:47600 | NULL | Binlog Dump | 4 | Master has sent all binlog to slave; waiting for binlog to be updated | NULL | 0 | 0 |
执行./bin/stop.sh后,发现442035连接未释放!!! ./log/90/90.log如下:
2016-01-14 11:55:11.655 [Thread-5] INFO c.a.otter.canal.instance.spring.CanalInstanceWithSpring - stop CannalInstance for 1-90 2016-01-14 11:55:11.670 [Thread-5] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - disConnect MysqlConnection to /192.168.12.110:3309... 2016-01-14 11:55:11.675 [destination = 90 , address = /192.168.12.110:3309 , EventParser] INFO c.a.o.canal.parse.inbound.mysql.dbsync.DirectLogFetcher - I/O interrupted while readi ng from client socket java.nio.channels.ClosedByInterruptException: null at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) ~[na:1.7.0_55] at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:412) ~[na:1.7.0_55] at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.DirectLogFetcher.fetch0(DirectLogFetcher.java:154) ~[canal.parse-1.0.21-SNAPSHOT.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.DirectLogFetcher.fetch(DirectLogFetcher.java:70) ~[canal.parse-1.0.21-SNAPSHOT.jar:na] at com.alibaba.otter.canal.parse.inbound.mysql.MysqlConnection.dump(MysqlConnection.java:113) [canal.parse-1.0.21-SNAPSHOT.jar:na] at com.alibaba.otter.canal.parse.inbound.AbstractEventParser$3.run(AbstractEventParser.java:209) [canal.parse-1.0.21-SNAPSHOT.jar:na] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_55] 2016-01-14 11:55:11.676 [destination = 90 , address = /192.168.12.110:3309 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - the channel /192.168.12.110 :3309 is not connected 2016-01-14 11:55:11.676 [destination = 90 , address = /192.168.12.110:3309 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - disConnect MysqlConnection to /192.168.12.110:3309... 2016-01-14 11:55:11.677 [Thread-5] INFO c.a.otter.canal.instance.spring.CanalInstanceWithSpring - stop successful.... 2016-01-14 11:55:11.677 [Thread-5] INFO c.a.otter.canal.server.embedded.CanalServerWithEmbedded - stop CanalInstances[90] successfully
另外,偶尔还会出现以下莫名mysql连接:
原提问者GitHub用户GitHub-BruceTong
经过debug测试,canal已经正确关闭了socket,只不过部分情况下操作系统的tcp链接会一直处于close_wait状态,可以通过调整操作系统参数,加速回收这类状态的链接.
mysql binlogdump为服务端主动推送数据模式,在客户端发起socket.close之后,服务端一致为发送ACK包导致一致处于CLOSE_WAIT状态.
采取的策略:针对binlog dump链接,在socket.close完成之后,再新开一个链接发起一次KILL CONNECTION ID请求彻底关闭binlogdump链接.
原回答者GitHub用户agapple
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。