大数据计算MaxCompute今天出现批量任务挂掉的情况，帮忙看下？
大数据计算MaxCompute今天出现批量任务挂掉的情况，帮忙看下，报错信息如下：org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:591)
at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:326)
at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:229)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:201)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:185)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:235)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.kafka.connect.errors.ConnectException: org.apache.kafka.connect.errors.ConnectException: RequestId=20231031095938d3d9c20b0cd9bcf3, ErrorCode=InternalServerError, ErrorMessage=Service internal error, please try again later.
at io.confluent.connect.odps.sink.OdpsSinkTask.put(OdpsSinkTask.java:109)
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:563)
... 10 more
Caused by: org.apache.kafka.connect.errors.ConnectException: RequestId=20231031095938d3d9c20b0cd9bcf3, ErrorCode=InternalServerError, ErrorMessage=Service internal error, please try again later.
at io.confluent.connect.odps.util.CachedConnectionProvider.getConnection(CachedConnectionProvider.java:73)
at io.confluent.connect.odps.sink.OdpsWriter.write(OdpsWriter.java:98)
at io.confluent.connect.odps.sink.OdpsSinkTask.put(OdpsSinkTask.java:80)
... 11 more
Caused by: RequestId=20231031095938d3d9c20b0cd9bcf3, ErrorCode=InternalServerError, ErrorMessage=Service internal error, please try again later.
at com.aliyun.odps.tunnel.StreamUploadSessionImpl.initiate(StreamUploadSessionImpl.java:81)
at com.aliyun.odps.tunnel.StreamUploadSessionImpl.(StreamUploadSessionImpl.java:49)
at com.aliyun.odps.tunnel.TableTunnel.createStreamUploadSession(TableTunnel.java:684)
at com.aliyun.odps.tunnel.TableTunnel.createStreamUploadSession(TableTunnel.java:673)
at io.confluent.connect.odps.dialect.OdpsStreamDialect.getConnection(OdpsStreamDialect.java:73)
at io.confluent.connect.odps.util.CachedConnectionProvider.newConnection(CachedConnectionProvider.java:97)
at io.confluent.connect.odps.util.CachedConnectionProvider.getConnection(CachedConnectionProvider.java:62)
2023-10-31 02:29:42,556 INFO || Attempting to open connection #1 to io.confluent.connect.odps.dialect.OdpsStreamDialect@56e4e5e6 [io.confluent.connect.odps.util.CachedConnectionProvider]
2023-10-31 02:29:43,463 INFO || Unable to connect to database on attempt 1/3. Will retry in 10000 ms. [io.confluent.connect.odps.util.CachedConnectionProvider]
RequestId=20231031102942d61569640cdbe987, ErrorCode=InternalServerError, ErrorMessage=Service internal error, please try again later.
at com.aliyun.odps.tunnel.StreamUploadSessionImpl.initiate(StreamUploadSessionImpl.java:81)
at com.aliyun.odps.tunnel.StreamUploadSessionImpl.(StreamUploadSessionImpl.java:49)
at com.aliyun.odps.tunnel.TableTunnel.createStreamUploadSession(TableTunnel.java:684)
at com.aliyun.odps.tunnel.TableTunnel.createStreamUploadSession(TableTunnel.java:673)
at io.confluent.connect.odps.dialect.OdpsStreamDialect.getConnection(OdpsStreamDialect.java:73)
at io.confluent.connect.odps.util.CachedConnectionProvider.newConnection(CachedConnectionProvider.java:97)
at io.confluent.connect.odps.util.CachedConnectionProvider.getConnection(CachedConnectionProvider.java:62)
at io.confluent.connect.odps.sink.OdpsWriter.write(OdpsWriter.java:98)
at io.confluent.connect.odps.sink.OdpsSinkTask.put(OdpsSinkTask.java:80)
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:563)
at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:326)
at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:229)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:201)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:185)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:235)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
写回答
-
从报错信息来看，这是一个Kafka Connect的批量任务挂掉的问题。具体原因可能是由于MaxCompute服务内部错误导致的。
2023-11-06 11:28:20赞同 展开评论 打赏 -
面对过去，不要迷离；面对未来，不必彷徨；活在今天，你只要把自己完全展示给别人看。这段错误日志显示了在尝试将数据从Kafka流式传输到MaxCompute表时出现的一个错误。根据错误消息，看起来这是一个“请求内部服务器错误”。这意味着MaxCompute服务器端出现了问题，可能是由于网络连接问题或其他因素导致的。
在解决这个问题之前，建议先检查MaxCompute表的权限设置是否正确，以及MaxCompute服务器的运行状态是否正常。此外，可以尝试清理和重建连接缓存，或者重启应用程序来解决问题。2023-11-05 16:11:32赞同 展开评论 打赏
版权声明：本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。
MaxCompute（原ODPS）是一项面向分析的大数据计算服务，它以Serverless架构提供快速、全托管的在线数据仓库服务，消除传统数据平台在资源扩展性和弹性方面的限制，最小化用户运维投入，使您经济并高效的分析处理海量数据。
