如下,我使用到了Guava的BloomFilter,貌似是基于kyro序列化的。检查点成功了,但是基于检查点恢复任务是失败的。报错堆栈如下,关键错误是什么无法访问public修饰的成员? java.lang.Exception: Exception while creating StreamOperatorStateContext. at org.apache.flink.streaming.api.operators. StreamTaskStateInitializerImpl.streamOperatorStateContext( StreamTaskStateInitializerImpl.java:235) at org.apache.flink.streaming.api.operators.AbstractStreamOperator .initializeState(AbstractStreamOperator.java:248) at org.apache.flink.streaming.runtime.tasks.OperatorChain .initializeStateAndOpenOperators(OperatorChain.java:400) at org.apache.flink.streaming.runtime.tasks.StreamTask .lambda$beforeInvoke$2(StreamTask.java:507) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1 .runThrowing(StreamTaskActionExecutor.java:47) at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke( StreamTask.java:501) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask .java:531) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.flink.util.FlinkException: Could not restore keyed state backend for KeyedProcessOperator_efc74365c4197a3ac1978016263fc7e7_(5/ 30) from any of the 1 provided restore options. at org.apache.flink.streaming.api.operators.BackendRestorerProcedure .createAndRestore(BackendRestorerProcedure.java:135) at org.apache.flink.streaming.api.operators. StreamTaskStateInitializerImpl.keyedStatedBackend( StreamTaskStateInitializerImpl.java:316) at org.apache.flink.streaming.api.operators. StreamTaskStateInitializerImpl.streamOperatorStateContext( StreamTaskStateInitializerImpl.java:155) ... 9 more Caused by: org.apache.flink.runtime.state.BackendBuildingException: Failed when trying to restore heap backend at org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder .build(HeapKeyedStateBackendBuilder.java:116) at org.apache.flink.runtime.state.filesystem.FsStateBackend .createKeyedStateBackend(FsStateBackend.java:540) at org.apache.flink.runtime.state.filesystem.FsStateBackend .createKeyedStateBackend(FsStateBackend.java:100) at org.apache.flink.runtime.state.StateBackend.createKeyedStateBackend( StateBackend.java:178) at org.apache.flink.streaming.api.operators. StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1( StreamTaskStateInitializerImpl.java:299) at org.apache.flink.streaming.api.operators.BackendRestorerProcedure .attemptCreateAndRestore(BackendRestorerProcedure.java:142) at org.apache.flink.streaming.api.operators.BackendRestorerProcedure .createAndRestore(BackendRestorerProcedure.java:121) ... 11 more Caused by: com.esotericsoftware.kryo.KryoException: Error constructing instance of class: com.google.common.hash.LongAdder Serialization trace: bitCount (com.google.common.hash.BloomFilterStrategies$LockFreeBitArray) bits (com.google.common.hash.BloomFilter) at com.twitter.chill.Instantiators$$anon$1.newInstance(KryoBase.scala: 136) at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1061) at com.esotericsoftware.kryo.serializers.FieldSerializer.create( FieldSerializer.java:547) at com.esotericsoftware.kryo.serializers.FieldSerializer.read( FieldSerializer.java:523) at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField .java:106) at com.esotericsoftware.kryo.serializers.FieldSerializer.read( FieldSerializer.java:528) at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:730) at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField .java:113) at com.esotericsoftware.kryo.serializers.FieldSerializer.read( FieldSerializer.java:528) at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) at com.esotericsoftware.kryo.serializers.MapSerializer.read( MapSerializer.java:143) at com.esotericsoftware.kryo.serializers.MapSerializer.read( MapSerializer.java:21) at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer .deserialize(KryoSerializer.java:346) at org.apache.flink.runtime.state.heap.StateTableByKeyGroupReaders .lambda$createV2PlusReader$0(StateTableByKeyGroupReaders.java:77) at org.apache.flink.runtime.state. KeyGroupPartitioner$PartitioningResultKeyGroupReader.readMappingsInKeyGroup( KeyGroupPartitioner.java:297) at org.apache.flink.runtime.state.heap.HeapRestoreOperation .readKeyGroupStateData(HeapRestoreOperation.java:299) at org.apache.flink.runtime.state.heap.HeapRestoreOperation .readStateHandleStateData(HeapRestoreOperation.java:260) at org.apache.flink.runtime.state.heap.HeapRestoreOperation.restore( HeapRestoreOperation.java:160) at org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder .build(HeapKeyedStateBackendBuilder.java:114) ... 17 more Caused by: java.lang.IllegalAccessException: Class com.twitter.chill. Instantiators$$anonfun$normalJava$1 can not access a member of class com.google.common.hash.LongAdder with modifiers "public" at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102) at java.lang.reflect.AccessibleObject.slowCheckMemberAccess( AccessibleObject.java:296) at java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java: 288) at java.lang.reflect.Constructor.newInstance(Constructor.java:413) at com.twitter.chill.Instantiators$$anonfun$normalJava$1.apply(KryoBase .scala:170) at com.twitter.chill.Instantiators$$anon$1.newInstance(KryoBase.scala: 133) ... 37 more*来自志愿者整理的flink邮件归档
自定义 state 的 type-serializer,可以尝试下 SimpleVersionedSerializer/avro/protobuf, etc. 复杂状态应该尽量避免使用依赖反射的 serializer,java Serializable 同理,这些都不好做 state upgrade。*来自志愿者整理的flink邮件归档
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。