启动canal-server后报错 can't find destination

环境信息

canal version 1.1.3 mysql version 5.6

问题描述

canal.properties文件的内容

canal.id = 1 canal.ip = canal.port = 11111 canal.metrics.pull.port = 11112 canal.zkServers = canal.zookeeper.flush.period = 1000 canal.withoutNetty = false canal.serverMode = tcp canal.file.data.dir = ${canal.conf.dir} canal.file.flush.period = 1000 canal.instance.memory.buffer.size = 16384 canal.instance.memory.buffer.memunit = 1024 canal.instance.memory.batch.mode = MEMSIZE canal.instance.memory.rawEntry = true canal.instance.detecting.enable = false canal.instance.detecting.sql = select 1 canal.instance.detecting.interval.time = 3 canal.instance.detecting.retry.threshold = 3 canal.instance.detecting.heartbeatHaEnable = false canal.instance.transaction.size = 1024 canal.instance.fallbackIntervalInSeconds = 60 canal.instance.network.receiveBufferSize = 16384 canal.instance.network.sendBufferSize = 16384 canal.instance.network.soTimeout = 30 canal.instance.filter.druid.ddl = true canal.instance.filter.query.dcl = false canal.instance.filter.query.dml = false canal.instance.filter.query.ddl = false canal.instance.filter.table.error = false canal.instance.filter.rows = false canal.instance.filter.transaction.entry = false canal.instance.binlog.format = ROW,STATEMENT,MIXED canal.instance.binlog.image = FULL,MINIMAL,NOBLOB canal.instance.get.ddl.isolation = false canal.instance.parser.parallel = true canal.instance.parser.parallelBufferSize = 256 canal.instance.tsdb.enable = true canal.instance.tsdb.dir = ${canal.file.data.dir:../conf}/${canal.instance.destination:} canal.instance.tsdb.url = jdbc:h2:${canal.instance.tsdb.dir}/h2;CACHE_SIZE=1000;MODE=MYSQL; canal.instance.tsdb.dbUsername = canal canal.instance.tsdb.dbPassword = canal canal.instance.tsdb.snapshot.interval = 24 canal.instance.tsdb.snapshot.expire = 360 canal.aliyun.accesskey = canal.aliyun.secretkey = canal.destinations = example canal.conf.dir = ../conf canal.auto.scan = true canal.auto.scan.interval = 5 canal.instance.tsdb.spring.xml = classpath:spring/tsdb/h2-tsdb.xml canal.instance.global.mode = spring canal.instance.global.lazy = false canal.instance.global.spring.xml = classpath:spring/file-instance.xml canal.mq.servers = 127.0.0.1:6667 canal.mq.retries = 0 canal.mq.batchSize = 16384 canal.mq.maxRequestSize = 1048576 canal.mq.lingerMs = 1 canal.mq.bufferMemory = 33554432 canal.mq.canalBatchSize = 50 canal.mq.canalGetTimeout = 100 canal.mq.flatMessage = true canal.mq.compressionType = none canal.mq.acks = all

instance.properties 文件的内容

canal.instance.mysql.slaveId=1234 canal.instance.gtidon=false canal.instance.master.address=192.168.23.170:25022 canal.instance.master.journal.name= canal.instance.master.position= canal.instance.master.timestamp= canal.instance.master.gtid= canal.instance.rds.accesskey= canal.instance.rds.secretkey= canal.instance.rds.instanceId= canal.instance.tsdb.enable=true canal.instance.dbUsername=canal canal.instance.dbPassword=canal canal.instance.connectionCharset = UTF-8 canal.instance.defaultDatabaseName =wenzhou canal.instance.enableDruid=false canal.instance.filter.regex=... canal.instance.filter.black.regex= canal.mq.topic=example canal.mq.partition=0

application.yml 文件内容 server: port: 8081 logging: level: org.springframework: WARN com.alibaba.otter.canal.client.adapter.hbase: DEBUG com.alibaba.otter.canal.client.adapter.es: DEBUG com.alibaba.otter.canal.client.adapter.rdb: DEBUG spring: jackson: date-format: yyyy-MM-dd HH:mm:ss time-zone: GMT+8 default-property-inclusion: non_null

canal.conf: canalServerHost: 127.0.0.1:11111 batchSize: 500 syncBatchSize: 1000 retries: 0 timeout: mode: tcp srcDataSources: defaultDS: url: jdbc:mysql://192.168.23.170:25022/wenzhou?useUnicode=true username: canal password: canal canalAdapters:

instance: output groups: groupId: g1 outerAdapters: name: es hosts: 192.168.10.229:9200 # es 集群地址, 逗号分隔 properties: cluster.name: elasticsearch # es cluster name estest.yml 往es输出

dataSourceKey: defaultDS destination: output esMapping: _index: mytest_user _type: _doc _id: _id

pk: id sql: "SELECT a.forderid AS _forderid,a.fpusername AS _fpusername,a.funitprice AS _funitprice,a.faddressdetail AS _faddressdetaial FROM t_groupon_childorder a"

objFields: _labels: array:; etlCondition: "where a.c_time>='{0}'" commitBatch: 3000

Steps to reproduce Expected behaviour 启动canal-deployer后会canal.log中出现报错 2019-03-12 18:00:06.833 [New I/O server worker #1-8] ERROR c.a.o.c.common.zookeeper.running.ServerRunningMonitor - start failed com.google.common.collect.ComputationException: com.alibaba.otter.canal.server.exception.CanalServerException: can't find destination:{}

example.log的日志

2019-03-12 17:48:59.564 [main] INFO c.a.o.c.i.spring.support.PropertyPlaceholderConfigurer - Loading properties file from class path resource [canal.properties] 2019-03-12 17:48:59.568 [main] INFO c.a.o.c.i.spring.support.PropertyPlaceholderConfigurer - Loading properties file from class path resource [example/instance.properties] 2019-03-12 17:48:59.735 [main] WARN o.s.beans.GenericTypeAwarePropertyDescriptor - Invalid JavaBean property 'connectionCharset' being accessed! Ambiguous write methods found next to actually used [public void com.alibaba.otter.canal.parse.inbound.mysql.AbstractMysqlEventParser.setConnectionCharset(java.nio.charset.Charset)]: [public void com.alibaba.otter.canal.parse.inbound.mysql.AbstractMysqlEventParser.setConnectionCharset(java.lang.String)] 2019-03-12 17:48:59.784 [main] INFO c.a.o.c.i.spring.support.PropertyPlaceholderConfigurer - Loading properties file from class path resource [canal.properties] 2019-03-12 17:48:59.784 [main] INFO c.a.o.c.i.spring.support.PropertyPlaceholderConfigurer - Loading properties file from class path resource [example/instance.properties] 2019-03-12 17:48:59.977 [main] ERROR com.alibaba.druid.pool.DruidDataSource - testWhileIdle is true, validationQuery not set 2019-03-12 17:49:00.270 [main] INFO c.a.otter.canal.instance.spring.CanalInstanceWithSpring - start CannalInstance for 1-example 2019-03-12 17:49:00.278 [main] WARN c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table filter : ^...$ 2019-03-12 17:49:00.278 [main] WARN c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table black filter : 2019-03-12 17:49:00.285 [main] INFO c.a.otter.canal.instance.core.AbstractCanalInstance - start successful.... 2019-03-12 17:49:00.354 [destination = example , address = /192.168.23.170:25022 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> begin to find start position, it will be long time for reset or first position 2019-03-12 17:49:00.354 [destination = example , address = /192.168.23.170:25022 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - prepare to find start position just show master status 2019-03-12 17:49:01.077 [destination = example , address = /192.168.23.170:25022 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> find start position successfully, EntryPosition[included=false,journalName=mysqld-bin.000008,position=299207,serverId=1,gtid=,timestamp=1552383741000] cost : 703ms , the next step is binlog dump

adapter.log日志报错信息

com.alibaba.otter.canal.protocol.exception.CanalClientException: failed to subscribe with reason: something goes wrong with channel:[id: 0x2f1cfd44, /127.0.0.1:45014 => /127.0.0.1:11111], exception=com.alibaba.otter.canal.server.exception.CanalServerException: destination:output should start first

找到adapter中application.yml canalAdapters:

instance: xxxx 以及deployer下canal.properties的 canal.destinations = wenzhou 是否对应

原提问者GitHub用户wajika