线上一个ETL Job不能跑了，报异常，这里为了说明问题简化表结构：
|
1
2
3
4
|
hive>
desc
void_t;
OK
x
int
None
z void None
|
而
|
1
|
select
*
from
void_t
|
确实会抛出异常：
|
1
|
14
/
03
/
0201
:
28
:
58
ERROR CliDriver: Failed with exceptionjava.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: Errorevaluating x
|
看到这个异常很疑惑，和x字段有什么关系呢，继续看详细日志：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
|
java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating x
atorg.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:
150
)
atorg.apache.hadoop.hive.ql.Driver.getResults(Driver.java:
1412
)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:
271
)
atorg.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:
216
)
atorg.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:
413
)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:
756
)
atorg.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:
614
)
atsun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
atsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39
)
atsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25
)
atjava.lang.reflect.Method.invoke(Method.java:
597
)
atorg.apache.hadoop.util.RunJar.main(RunJar.java:
208
)
Causedby: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating x
atorg.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:
80
)
atorg.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:
502
)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:
832
)
atorg.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:
90
)
atorg.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:
502
)
atorg.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:
490
)
atorg.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:
136
)
...
11
more
Causedby: java.lang.RuntimeException: Internal error: no LazyObject
for
VOID
atorg.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyPrimitiveClass(LazyFactory.java:
119
)
atorg.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObject(LazyFactory.java:
155
)
at org.apache.hadoop.hive.serde2.lazy.LazyStruct.parse(LazyStruct.java:
108
)
atorg.apache.hadoop.hive.serde2.lazy.LazyStruct.getField(LazyStruct.java:
190
)
atorg.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldData(LazySimpleStructObjectInspector.java:
188
)
atorg.apache.hadoop.hive.serde2.objectinspector.DelegatedStructObjectInspector.getStructFieldData(DelegatedStructObjectInspector.java:
79
)
atorg.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.evaluate(ExprNodeColumnEvaluator.java:
98
)
atorg.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:
76
)
|
看到这个noLazyObject for VOID才知道原来问题出现在这里，也就是字段z上；查看ETL Job里的Query发现里面一个建表的语句用到了create table xxx as select null as z from xxx这样的方式，进而生成了一个VOID类型的字段，但是Hive本身却无法处理该字段，在jira里确实也有这么一个unresolved的Bug:HIVE-2615
Workaround也比较简单：1.先建表再insert select 2.在ctas时cast(null as <type>) z来强制指定类型.
本文转自MIKE老毕 51CTO博客，原文链接：http://blog.51cto.com/boylook/1365747，如需转载请自行联系原作者
