flink版本1.15
在程序中写了,一个子查询进行聚合统计,然后在这个子查询外面再套了一层聚合统计。这样程序就报错了,报错信息如下:
java.lang.RuntimeException: Could not instantiate generated class 'GroupAggsHandler$537'
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:74) ~[flink-table-runtime-1.15.0.jar:1.15.0]
at org.apache.flink.table.runtime.operators.aggregate.MiniBatchLocalGroupAggFunction.open(MiniBatchLocalGroupAggFunction.java:59) ~[flink-table-runtime-1.15.0.jar:1.15.0]
at org.apache.flink.table.runtime.operators.bundle.AbstractMapBundleOperator.open(AbstractMapBundleOperator.java:82) ~[flink-table-runtime-1.15.0.jar:1.15.0]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:107) ~[flink-dist-1.15.0.jar:1.15.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:700) ~[flink-dist-1.15.0.jar:1.15.0]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55) ~[flink-dist-1.15.0.jar:1.15.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:676) ~[flink-dist-1.15.0.jar:1.15.0]
at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:643) ~[flink-dist-1.15.0.jar:1.15.0]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:948) ~[flink-dist-1.15.0.jar:1.15.0]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:917) ~[flink-dist-1.15.0.jar:1.15.0]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741) ~[flink-dist-1.15.0.jar:1.15.0]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) ~[flink-dist-1.15.0.jar:1.15.0]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]
Caused by: org.apache.flink.util.FlinkRuntimeException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:94) ~[flink-table-runtime-1.15.0.jar:1.15.0]
at org.apache.flink.table.runtime.generated.GeneratedClass.compile(GeneratedClass.java:101) ~[flink-table-runtime-1.15.0.jar:1.15.0]
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:68) ~[flink-table-runtime-1.15.0.jar:1.15.0]
... 12 more
Caused by: org.apache.flink.shaded.guava30.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051) ~[atp-vat-flink-data-1.0-SNAPSHOT.jar:?]
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.get(LocalCache.java:3962) ~[atp-vat-flink-data-1.0-SNAPSHOT.jar:?]
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4859) ~[atp-vat-flink-data-1.0-SNAPSHOT.jar:?]
at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:92) ~[flink-table-runtime-1.15.0.jar:1.15.0]
at org.apache.flink.table.runtime.generated.GeneratedClass.compile(GeneratedClass.java:101) ~[flink-table-runtime-1.15.0.jar:1.15.0]
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:68) ~[flink-table-runtime-1.15.0.jar:1.15.0]
... 12 more
Caused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:107) ~[flink-table-runtime-1.15.0.jar:1.15.0]
at org.apache.flink.table.runtime.generated.CompileUtils.lambda$compile$0(CompileUtils.java:92) ~[flink-table-runtime-1.15.0.jar:1.15.0]
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4864) ~[atp-vat-flink-data-1.0-SNAPSHOT.jar:?]
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[atp-vat-flink-data-1.0-SNAPSHOT.jar:?]
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[atp-vat-flink-data-1.0-SNAPSHOT.jar:?]
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[atp-vat-flink-data-1.0-SNAPSHOT.jar:?]
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[atp-vat-flink-data-1.0-SNAPSHOT.jar:?]
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.get(LocalCache.java:3962) ~[atp-vat-flink-data-1.0-SNAPSHOT.jar:?]
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4859) ~[atp-vat-flink-data-1.0-SNAPSHOT.jar:?]
at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:92) ~[flink-table-runtime-1.15.0.jar:1.15.0]
at org.apache.flink.table.runtime.generated.GeneratedClass.compile(GeneratedClass.java:101) ~[flink-table-runtime-1.15.0.jar:1.15.0]
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:68) ~[flink-table-runtime-1.15.0.jar:1.15.0]
... 12 more
Caused by: org.codehaus.commons.compiler.CompileException: Line 12, Column 28: Cannot determine simple type name "org"
这种情况是在开启minibatch的情况下会报错,如果不加minibathc那么程序能正常执行,我在本地IDEA中都能成功,但是在集群上运行程序就会报错。请问这是什么问题呢?
这个错误信息表明在编译生成的Flink SQL执行逻辑时遇到了问题,具体是无法确定类型"org"的简单类型名称。这可能是由于子查询的复杂性或者与MiniBatch模式的兼容性问题导致的。在Flink 1.15版本中,MiniBatch模式是一种优化,用于减少状态更新的频率,提高性能,但它可能对某些查询结构有特定的要求。
根据错误信息,问题可能出在以下几点:
state.backend.finite-state-operator.mini-batch.enabled
和state.backend.finite-state-operator.mini-batch.interval
,确保它们是正确的,并且与你的查询兼容。建议尝试以下步骤来解决问题:
记得在调整查询或配置时,始终在与生产环境相同或尽可能接近的环境中进行测试。
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。