dataworks这种问题需要怎么解决吗?

dataworks这种问题需要怎么解决吗?看起来像是哪里的参数配置有问题。
TASK-MESSAGE:
FAILED: org.apache.kyuubi.KyuubiSQLException: org.apache.kyuubi.KyuubiSQLException: Error operating ExecuteStatement: java.lang.OutOfMemoryError: GC overhead limit exceeded

    at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69)
    at org.apache.kyuubi.engine.spark.operation.SparkOperation$$anonfun$onError$1.$anonfun$applyOrElse$1(SparkOperation.scala:189)
    at org.apache.kyuubi.Utils$.withLockRequired(Utils.scala:395)
    at org.apache.kyuubi.operation.AbstractOperation.withLockRequired(AbstractOperation.scala:51)
    at org.apache.kyuubi.engine.spark.operation.SparkOperation$$anonfun$onError$1.applyOrElse(SparkOperation.scala:177)
    at org.apache.kyuubi.engine.spark.operation.SparkOperation$$anonfun$onError$1.applyOrElse(SparkOperation.scala:172)
    at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:38)
    at org.apache.kyuubi.engine.spark.operation.ExecuteStatement.executeStatement(ExecuteStatement.scala:90)
    at org.apache.kyuubi.engine.spark.operation.ExecuteStatement$$anon$1.run(ExecuteStatement.scala:101)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded@DataWorks官方支持(DataWorks官方支持)

展开
收起
真的很搞笑 2024-08-05 11:45:08 90 分享 版权
1 条回答
写回答
取消 提交回答
  • 可以尝试加一点driver和exector的内存,例如, "spark.exector.memory":"4g" (高级配置中),如果还是不行 辛苦咨询一下emr同学帮忙按需 是否有可调整的参数,在dw界面配置 可以先在节点右侧的高级配置中设置单个节点试一下 ,此回答整理自钉群“DataWorks交流群(答疑@机器人)”

    2024-08-06 19:23:20
    赞同 展开评论

DataWorks基于MaxCompute/Hologres/EMR/CDP等大数据引擎,为数据仓库/数据湖/湖仓一体等解决方案提供统一的全链路大数据开发治理平台。

还有其他疑问?
咨询AI助理