开发者社区> 问答> 正文

flink 写hive decimal类型报错

具体如下: java.lang.NoSuchMethodError: org.apache.hadoop.hive.serde2.io.HiveDecimalWritable.serialize64(I)J

at org.apache.orc.impl.ColumnStatisticsImpl$Decimal64StatisticsImpl.updateDecimal(ColumnStatisticsImpl.java:1010)

at org.apache.orc.impl.writer.DecimalTreeWriter.writeBatch(DecimalTreeWriter.java:99)

at org.apache.orc.impl.writer.DecimalTreeWriter.writeBatch(DecimalTreeWriter.java:159)

at org.apache.orc.impl.writer.StructTreeWriter.writeRootBatch(StructTreeWriter.java:56)

at org.apache.orc.impl.WriterImpl.addRowBatch(WriterImpl.java:557)

at org.apache.flink.orc.writer.OrcBulkWriter.addElement(OrcBulkWriter.java:58)

at org.apache.flink.table.filesystem.FileSystemTableSink$ProjectionBulkFactory$1.addElement(FileSystemTableSink.java:589)

at org.apache.flink.table.filesystem.FileSystemTableSink$ProjectionBulkFactory$1.addElement(FileSystemTableSink.java:585)

at org.apache.flink.streaming.api.functions.sink.filesystem.BulkPartWriter.write(BulkPartWriter.java:48)

at org.apache.flink.streaming.api.functions.sink.filesystem.Bucket.write(Bucket.java:209)

at org.apache.flink.streaming.api.functions.sink.filesystem.Buckets.onElement(Buckets.java:290)

at org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSinkHelper.onElement(StreamingFileSinkHelper.java:104)

at org.apache.flink.table.filesystem.stream.AbstractStreamingWriter.processElement(AbstractStreamingWriter.java:140)

at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:71)

at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:46)

at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:26)

at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:52)

at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:30)

at org.apache.flink.streaming.api.operators.StreamFilter.processElement(StreamFilter.java:40)

at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:71)

at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:46)

at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:26)

at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:52)

at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:30)

at StreamExecCalc$154.processElement(Unknown Source) 用的是flink-sql-connector-hive-2.3.6_2.11-1.12-SNAPSHOT.jar,公司的Hive也是这个版本,可能是什么原因导致?*来自志愿者整理的flink邮件归档

展开
收起
JACKJACK 2021-12-08 16:13:52 1641 0
1 条回答
写回答
取消 提交回答
  • 对的是我!

    你好,有设置过table.exec.hive.fallback-mapred-writer参数么?可以把它设置成true再试试。 *来自志愿者整理的flink邮件归档

    2021-12-08 16:54:47
    赞同 展开评论 打赏
问答排行榜
最热
最新

相关电子书

更多
Hive Bucketing in Apache Spark 立即下载
spark替代HIVE实现ETL作业 立即下载
2019大数据技术公开课第五季—Hive迁移到MaxCompute最佳实践 立即下载