开发者社区> 问答> 正文

Task not found&&增量同步es

elasticsearch version 7.6.2 canal version 1.1.6

配置文件:

canalAdapters: - instance: pickbean groups: - groupId: g1 outerAdapters: - name: logger - name: es7 key: pickbeanKey hosts: 127.0.0.1:9300 properties: mode: transport cluster.name: elasticsearch

dataSourceKey: defaultDS outerAdapterKey: pickbeanKey destination: pickbean
groupId: g1 esMapping: _index: material _id: _id upsert: true sql: "SELECT material_id AS _id, FROM material"
commitBatch: 3000

curl http://localhost:8081/etl/es7/pickbeanKey/material.yml -X POST error: {"succeeded":false,"errorMessage":"Task not found"}

日志文件:

2022-07-20 10:08:56.092 [http-nio-8081-exec-1] INFO o.a.catalina.core.ContainerBase.[Tomcat].[localhost].[/] - Initializing Spring FrameworkServlet 'dispatcherServlet' 2022-07-20 10:08:56.093 [http-nio-8081-exec-1] INFO org.springframework.web.servlet.DispatcherServlet - FrameworkServlet 'dispatcherServlet': initialization started 2022-07-20 10:08:56.127 [http-nio-8081-exec-1] INFO org.springframework.web.servlet.DispatcherServlet - FrameworkServlet 'dispatcherServlet': initialization completed in 33 ms

当我手动更新数据库时,监控是对的,es数据仍未更新,因为有task not fund问题怀疑是adapter配置错了:

2022-07-20 10:08:06.134 [Thread-2] INFO c.a.otter.canal.adapter.launcher.loader.AdapterProcessor - =============> Subscribe destination: pickbean succeed <============= 2022-07-20 10:08:37.035 [pool-1-thread-1] INFO c.a.o.canal.client.adapter.logger.LoggerAdapterExample - DML: {"database":"","destination":"pickbean","es":1658282916000,"groupId":"g1","isDdl":false,"old":[{"":""}],"pkNames":[""],"sql":"","table":"","ts":1658282916870,"type":"UPDATE"}

百度的解决方案:

1.加outerAdapterKey,已经尝试无效 2. 采用client-adapter.es7x-1.1.6-SNAPSHOT-jar-with-dependencies.jar替换原有的,已经尝试无效

目前我该如何处理,转成1.1.5版本吗,我看目前大部分的都是这个版本

原提问者GitHub用户armyhcz

展开
收起
白夜行fighting 2023-04-25 21:02:19 377 0
1 条回答
写回答
取消 提交回答
  • 已解决 问题原因:sql错误

    ERROR c.a.o.canal.adapter.launcher.loader.CanalAdapterLoader - Load canal adapter: es7 failed java.lang.RuntimeException: java.lang.RuntimeException: com.alibaba.druid.sql.parser.ParserException Caused by: com.alibaba.druid.sql.parser.ParserException: null

    解决方案: 关联查询的on a.id=b.id,需要至少有一个写入select中,我设置了别名a.id as _id,这时候需要把b.id写到select中 提供方案的文章:https://developer.aliyun.com/ask/310320?accounttraceid=be06dbd8a9004bdda40f91dfe5920cc9emjg 文章中详细指出代码报错行数,以及报错null原因

    原回答者GitHub用户armyhcz

    2023-04-26 17:47:18
    赞同 展开评论 打赏
问答排行榜
最热
最新

相关电子书

更多
低代码开发师(初级)实战教程 立即下载
冬季实战营第三期:MySQL数据库进阶实战 立即下载
阿里巴巴DevOps 最佳实践手册 立即下载