环境信息

canal version 1.1.3 mysql version 5.7.20 es version 6.5.0

问题描述

修改mysql 行记录数据，adapter 有监测到数据变更，但并没有同步到es，控制台打印如下日志：

2019-02-15 16:58:38.947 INFO 51934 --- [pool-3-thread-1] c.a.o.c.c.a.logger.LoggerAdapterExample : DML: {"data":null,"database":"inner3","destination":"example","es":1550221118000,"isDdl":null,"old":null,"pkNames":[],"sql":"/* ApplicationName=DataGrip 2017.2.2 / UPDATE inner3.tb_space t SET t.space_name = '哈哈大笑1234' WHERE t.id = 1008612772","table":"tb_space","ts":1550221118947,"type":"UPDATE"} 2019-02-15 16:58:38.948 DEBUG 51934 --- [pool-3-thread-1] c.a.o.c.c.a.es.service.ESSyncService : DML: {"data":null,"database":"inner3","destination":"example","es":1550221118000,"isDdl":null,"old":null,"pkNames":[],"sql":"/ ApplicationName=DataGrip 2017.2.2 */ UPDATE inner3.tb_space t SET t.space_name = '哈哈大笑1234' WHERE t.id = 1008612772","table":"tb_space","ts":1550221118948,"type":"UPDATE"}

具体配置文件：

application.yml

server: port: 8081 logging: level: org.springframework: INFO com.alibaba.otter.canal.client.adapter.hbase: DEBUG com.alibaba.otter.canal.client.adapter.es: DEBUG com.alibaba.otter.canal.client.adapter.rdb: DEBUG spring: jackson: date-format: yyyy-MM-dd HH:mm:ss time-zone: GMT+8 default-property-inclusion: non_null

canal.conf: canalServerHost: 127.0.0.1:11111 batchSize: 1 syncBatchSize: 1 retries: 0 timeout: accessKey: secretKey: mode: tcp # kafka rocketMQ srcDataSources: defaultDS: url: jdbc:mysql://127.0.0.1:3307/inner3?useUnicode=true username: root password: 123456 canalAdapters: - instance: example groups: - groupId: adapter1 outerAdapters: - name: logger - name: es hosts: 127.0.0.1:9300 properties: cluster.name: elasticsearch_reid

inner_space.yml

dataSourceKey: defaultDS destination: example esMapping: _index: inner_space _type: space _id: _id

pk: id

sql: "select id as _id, space_name as spaceName from tb_space"

objFields:

_labels: array:;

etlCondition: "where id >= '{0}'" commitBatch: 10

syncSwitch 已打开

2019-02-18 15:01:37.629 INFO 97397 --- [pool-3-thread-1] c.a.o.c.c.a.logger.LoggerAdapterExample : DML: {"data":null,"database":"inner3","destination":"example","es":1550473297000,"isDdl":null,"old":null,"pkNames":[],"sql":"/* ApplicationName=DataGrip 2017.2.2 / UPDATE inner3.tb_friend t SET t.target_id = 789 WHERE t.id = 17","table":"tb_friend","ts":1550473297629,"type":"UPDATE"} 2019-02-18 15:01:37.629 TRACE 97397 --- [pool-3-thread-1] c.a.o.c.c.a.es.service.ESSyncService : Destination: example, database:inner3, table:tb_friend, type:UPDATE, effect index count: 1 2019-02-18 15:01:37.629 TRACE 97397 --- [pool-3-thread-1] c.a.o.c.c.a.es.service.ESSyncService : Prepared to sync index: inner_friend, destination: example 2019-02-18 15:01:37.629 TRACE 97397 --- [pool-3-thread-1] c.a.o.c.c.a.es.service.ESSyncService : Sync elapsed time: 0 ms,destination: example, es index: inner_friend 2019-02-18 15:01:37.629 TRACE 97397 --- [pool-3-thread-1] c.a.o.c.c.a.es.service.ESSyncService : Sync completed: inner_friend, destination: example 2019-02-18 15:01:37.629 TRACE 97397 --- [pool-3-thread-1] c.a.o.c.c.a.es.service.ESSyncService : Sync elapsed time: 0 ms, effect index count：1, destination: example 2019-02-18 15:01:37.630 DEBUG 97397 --- [pool-3-thread-1] c.a.o.c.c.a.es.service.ESSyncService : DML: {"data":null,"database":"inner3","destination":"example","es":1550473297000,"isDdl":null,"old":null,"pkNames":[],"sql":"/ ApplicationName=DataGrip 2017.2.2 */ UPDATE inner3.tb_friend t SET t.target_id = 789 WHERE t.id = 17","table":"tb_friend","ts":1550473297629,"type":"UPDATE"}

显示同步成功，但数据并没更新到 es

原提问者GitHub用户linyongfu2013