MaxCompute操作报错合集之DataWorks在绑定MaxCompute引擎时，报错，如何解决-阿里云开发者社区

问题一：DataWorks绑定maxcompute引擎报这个错误，可能是啥问题啊？

DataWorks绑定maxcompute引擎报这个错误，可能是啥问题啊？

参考答案：

这个名字是您创建maxcompute数据源的时候手动取的吗老版本的逻辑绑定默认会生成一个叫odps first的数据源

关于本问题的更多回答可点击进行查看：

https://developer.aliyun.com/ask/574807

问题二：DataWorks采用阿里flink写入maxcompute，拉取依赖的时候报错了？

DataWorks采用阿里flink写入maxcompute，拉取依赖的时候报错了？

这个是什么问题呀？

参考答案：

这个错误提示表示在拉取依赖时，找不到名为'com.aliyun.odps:flink-connector-odps:iar:113.0'的依赖。请检查以下几点：

确保您的项目中已经添加了正确的依赖。在项目的pom.xml文件中添加以下依赖：

<dependency>
    <groupId>com.aliyun.odps</groupId>
    <artifactId>flink-connector-odps</artifactId>
    <version>iar:113.0</version>
</dependency>

如果您使用的是Maven，请确保您的本地仓库中存在该依赖。您可以通过运行mvn clean install命令来下载并安装依赖。
如果问题仍然存在，尝试将版本号更改为最新版本，例如：

<dependency>
    <groupId>com.aliyun.odps</groupId>
    <artifactId>flink-connector-odps</artifactId>
    <version>iar:latest</version>
</dependency>

如果以上方法都无法解决问题，请检查您的网络连接和防火墙设置，确保您可以访问阿里云的Maven仓库。

关于本问题的更多回答可点击进行查看：

https://developer.aliyun.com/ask/574773

问题三：开源spark3.1.3结构化流写maxcompute报错

当我使用https://github.com/aliyun/aliyun-maxcompute-data-collectors/tree/master/spark-datasource-v3.1 中开源的spark连接器往maxcompute写数据时会在固定时间段报错，白天可以正常写入数据，但是到凌晨有一定概率会报错，提示报错如下：

Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 32.0 failed 4 times, most recent failure: Lost task 1.3 in stage 32.0 (TID 130) (10.233.122.167 executor 1): java.net.SocketException: Unexpected end of file from server

at java.base/sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)

at java.base/sun.net.www.http.HttpClient.parseHTTP(Unknown Source)

at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)

at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)

at java.base/java.net.HttpURLConnection.getResponseCode(Unknown Source)

at com.aliyun.odps.commons.transport.DefaultConnection.getResponse(DefaultConnection.java:132)

at com.aliyun.odps.tunnel.io.TunnelRecordWriter.write(TunnelRecordWriter.java:75)

at com.aliyun.odps.cupid.table.v1.tunnel.impl.TunnelWriter.write(TunnelWriter.java:62)

at com.aliyun.odps.cupid.table.v1.tunnel.impl.TunnelWriter.write(TunnelWriter.java:19)

at org.apache.spark.sql.odps.writer.DynamicPartitionWriter.write(DynamicPartitionWriter.scala:47)

at org.apache.spark.sql.odps.writer.DynamicPartitionWriter.write(DynamicPartitionWriter.scala:30)

at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.$anonfun$run$1(WriteToDataSourceV2Exec.scala:416)

at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1473)

at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:452)

at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:360)

at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)

at org.apache.spark.scheduler.Task.run(Task.scala:131)

at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:498)

at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)

at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:501)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.base/java.lang.Thread.run(Unknown Source)

Suppressed: java.io.IOException: Stream is closed

at java.base/sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.checkError(Unknown Source)

at java.base/sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(Unknown Source)

at java.base/java.util.zip.DeflaterOutputStream.deflate(Unknown Source)

at java.base/java.util.zip.DeflaterOutputStream.write(Unknown Source)

at org.apache.commons.io.output.ProxyOutputStream.write(ProxyOutputStream.java:90)

at com.google.protobuf.CodedOutputStream.refreshBuffer(CodedOutputStream.java:833)

at com.google.protobuf.CodedOutputStream.writeRawByte(CodedOutputStream.java:892)

at com.google.protobuf.CodedOutputStream.writeRawByte(CodedOutputStream.java:900)

at com.google.protobuf.CodedOutputStream.writeRawVarint32(CodedOutputStream.java:1012)

at com.google.protobuf.CodedOutputStream.writeTag(CodedOutputStream.java:994)

at com.google.protobuf.CodedOutputStream.writeSInt64(CodedOutputStream.java:273)

at com.aliyun.odps.commons.proto.ProtobufRecordStreamWriter.close(ProtobufRecordStreamWriter.java:371)

at com.aliyun.odps.tunnel.io.TunnelRecordWriter.close(TunnelRecordWriter.java:85)

at com.aliyun.odps.cupid.table.v1.tunnel.impl.TunnelWriter.close(TunnelWriter.java:71)

at org.apache.spark.sql.odps.writer.DynamicPartitionWriter.abort(DynamicPartitionWriter.scala:62)

at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.$anonfun$run$6(WriteToDataSourceV2Exec.scala:448)

at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1484)

... 10 more

Suppressed: java.lang.NullPointerException: Deflater has been closed

at java.base/java.util.zip.Deflater.ensureOpen(Unknown Source)

at java.base/java.util.zip.Deflater.deflate(Unknown Source)