问题描述:
使用release-0.9.0分支对hudi进行打包,在使用flink以及spark的时候会报配置项空指针问题。就是某些配置项本来是可选的,有默认值的,但是在使用过程中发现该默认值无法加载。例如:
ction.scala:122) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) [flink-dist_2.12-1.12.2.jar:1.12.2]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.12-1.12.2.jar:1.12.2]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-dist_2.12-1.12.2.jar:1.12.2]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.actor.Actor.aroundReceive(Actor.scala:517) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.actor.Actor.aroundReceive$(Actor.scala:515) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.12-1.12.2.jar:1.12.2]
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.12-1.12.2.jar:1.12.2]
Caused by: org.apache.flink.util.FlinkRuntimeException: Failed to start the operator coordinators
at org.apache.flink.runtime.scheduler.SchedulerBase.startAllOperatorCoordinators(SchedulerBase.java:1258) ~[flink-dist_2.12-1.12.2.jar:1.12.2]
at org.apache.flink.runtime.scheduler.SchedulerBase.startScheduling(SchedulerBase.java:624) ~[flink-dist_2.12-1.12.2.jar:1.12.2]
at org.apache.flink.runtime.jobmaster.JobMaster.startScheduling(JobMaster.java:1032) ~[flink-dist_2.12-1.12.2.jar:1.12.2]
at java.util.concurrent.CompletableFuture.uniRun(CompletableFuture.java:719) ~[?:1.8.0_241]
... 29 more
Caused by: java.lang.NullPointerException: Name is null
at java.lang.Enum.valueOf(Enum.java:236) ~[?:1.8.0_241]
at org.apache.hudi.metrics.MetricsReporterType.valueOf(MetricsReporterType.java:24) ~[hudi-flink-bundle_2.12-0.9.0.jar:0.9.0]
at org.apache.hudi.config.HoodieMetricsConfig$Builder.build(HoodieMetricsConfig.java:262) ~[hudi-flink-bundle_2.12-0.9.0.jar:0.9.0]
at org.apache.hudi.config.HoodieWriteConfig$Builder.setDefaults(HoodieWriteConfig.java:2085) ~[hudi-flink-bundle_2.12-0.9.0.jar:0.9.0]
at org.apache.hudi.config.HoodieWriteConfig$Builder.build(HoodieWriteConfig.java:2120) ~[hudi-flink-bundle_2.12-0.9.0.jar:0.9.0]
at org.apache.hudi.util.StreamerUtil.getHoodieClientConfig(StreamerUtil.java:195) ~[hudi-flink-bundle_2.12-0.9.0.jar:0.9.0]
at org.apache.hudi.util.StreamerUtil.createWriteClient(StreamerUtil.java:317) ~[hudi-flink-bundle_2.12-0.9.0.jar:0.9.0]
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.start(StreamWriteOperatorCoordinator.java:153) ~[hudi-flink-bundle_2.12-0.9.0.jar:0.9.0]
at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder.start(OperatorCoordinatorHolder.java:189) ~[flink-dist_2.12-1.12.2.jar:1.12.2]
at org.apache.flink.runtime.scheduler.SchedulerBase.startAllOperatorCoordinators(SchedulerBase.java:1253) ~[flink-dist_2.12-1.12.2.jar:1.12.2]
at org.apache.flink.runtime.scheduler.SchedulerBase.startScheduling(SchedulerBase.java:624) ~[flink-dist_2.12-1.12.2.jar:1.12.2]
at org.apache.flink.runtime.jobmaster.JobMaster.startScheduling(JobMaster.java:1032) ~[flink-dist_2.12-1.12.2.jar:1.12.2]
at java.util.concurrent.CompletableFuture.uniRun(CompletableFuture.java:719) ~[?:1.8.0_241]
... 29 more
2021-09-13 05:15:11,626 INFO org.apache.flink.runtime.blob.BlobServer [] - Stopped BLOB server at 0.0.0.0:33119
修复方法
hudi在master对此问题进行了修复,具体可参考以下commit:
https://github.com/apache/hudi/commit/0f39137ba854a2f808e2108c6aa10f4260b00844
对源码进行修改即可。