elasticsearch version 7.6.2 canal version 1.1.6
配置文件:
canalAdapters: - instance: pickbean groups: - groupId: g1 outerAdapters: - name: logger - name: es7 key: pickbeanKey hosts: 127.0.0.1:9300 properties: mode: transport cluster.name: elasticsearch
dataSourceKey: defaultDS outerAdapterKey: pickbeanKey destination: pickbean
groupId: g1 esMapping: _index: material _id: _id upsert: true sql: "SELECT material_id
AS _id, FROM material"
commitBatch: 3000
curl http://localhost:8081/etl/es7/pickbeanKey/material.yml -X POST error: {"succeeded":false,"errorMessage":"Task not found"}
日志文件:
2022-07-20 10:08:56.092 [http-nio-8081-exec-1] INFO o.a.catalina.core.ContainerBase.[Tomcat].[localhost].[/] - Initializing Spring FrameworkServlet 'dispatcherServlet' 2022-07-20 10:08:56.093 [http-nio-8081-exec-1] INFO org.springframework.web.servlet.DispatcherServlet - FrameworkServlet 'dispatcherServlet': initialization started 2022-07-20 10:08:56.127 [http-nio-8081-exec-1] INFO org.springframework.web.servlet.DispatcherServlet - FrameworkServlet 'dispatcherServlet': initialization completed in 33 ms
当我手动更新数据库时,监控是对的,es数据仍未更新,因为有task not fund问题怀疑是adapter配置错了:
2022-07-20 10:08:06.134 [Thread-2] INFO c.a.otter.canal.adapter.launcher.loader.AdapterProcessor - =============> Subscribe destination: pickbean succeed <============= 2022-07-20 10:08:37.035 [pool-1-thread-1] INFO c.a.o.canal.client.adapter.logger.LoggerAdapterExample - DML: {"database":"","destination":"pickbean","es":1658282916000,"groupId":"g1","isDdl":false,"old":[{"":""}],"pkNames":[""],"sql":"","table":"","ts":1658282916870,"type":"UPDATE"}
百度的解决方案:
1.加outerAdapterKey,已经尝试无效 2. 采用client-adapter.es7x-1.1.6-SNAPSHOT-jar-with-dependencies.jar替换原有的,已经尝试无效
目前我该如何处理,转成1.1.5版本吗,我看目前大部分的都是这个版本
原提问者GitHub用户armyhcz
已解决 问题原因:sql错误
ERROR c.a.o.canal.adapter.launcher.loader.CanalAdapterLoader - Load canal adapter: es7 failed java.lang.RuntimeException: java.lang.RuntimeException: com.alibaba.druid.sql.parser.ParserException Caused by: com.alibaba.druid.sql.parser.ParserException: null
解决方案: 关联查询的on a.id=b.id,需要至少有一个写入select中,我设置了别名a.id as _id,这时候需要把b.id写到select中 提供方案的文章:https://developer.aliyun.com/ask/310320?accounttraceid=be06dbd8a9004bdda40f91dfe5920cc9emjg 文章中详细指出代码报错行数,以及报错null原因
原回答者GitHub用户armyhcz
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。