UnsupportedOperationException: Currently the writer can only accept BytesRefArrayWritable

简介: 往Hive表中插入时报错: java.lang.RuntimeException: java.lang.UnsupportedOperationException: Currently the writer can only accept BytesRefArrayWritable at org.

往Hive表中插入时报错:

java.lang.RuntimeException: java.lang.UnsupportedOperationException: Currently the writer can only accept BytesRefArrayWritable
	at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:270)
	at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:506)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:447)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.UnsupportedOperationException: Currently the writer can only accept BytesRefArrayWritable
	at org.apache.hadoop.hive.ql.io.RCFile$Writer.append(RCFile.java:880)
	at org.apache.hadoop.hive.ql.io.RCFileOutputFormat$2.write(RCFileOutputFormat.java:140)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:606)

java.lang.RuntimeException: java.lang.UnsupportedOperationException: Currently the writer can only accept BytesRefArrayWritable
	at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:270)
	at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:506)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:447)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.UnsupportedOperationException: Currently the writer can only accept BytesRefArrayWritable
	at org.apache.hadoop.hive.ql.io.RCFile$Writer.append(RCFile.java:880)
	at org.apache.hadoop.hive.ql.io.RCFileOutputFormat$2.write(RCFileOutputFormat.java:140)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:606)

java.lang.RuntimeException: java.lang.UnsupportedOperationException: Currently the writer can only accept BytesRefArrayWritable
	at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:270)
	at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:506)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:447)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.UnsupportedOperationException: Currently the writer can only accept BytesRefArrayWritable
	at org.apache.hadoop.hive.ql.io.RCFile$Writer.append(RCFile.java:880)
	at org.apache.hadoop.hive.ql.io.RCFileOutputFormat$2.write(RCFileOutputFormat.java:140)
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:606)

貌似建表是格式的问题:http://comments.gmane.org/gmane.comp.java.hadoop.hive.user/2849

建表语句如下:

CREATE  TABLE client_user_type_installtime(
  userkey string,
  mos string,
  type string)
PARTITIONED BY ( 
  dt string,
  installtime_type string)
ROW FORMAT DELIMITED 
  FIELDS TERMINATED BY '9' 
  LINES TERMINATED BY '10' 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.RCFileInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.RCFileOutputFormat'

修改建表语句:

CREATE  TABLE client_user_type_installtime(
  userkey string, 
  mos string, 
  type string)
PARTITIONED BY ( 
  dt string, 
  installtime_type string)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.mapred.TextInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'

重建后插入成功。

 

目录
相关文章
|
5月前
Request Headers 中的 Accept 是 text/event-stream
Request Headers 中的 Accept 是 text/event-stream
592 0
|
Java 容器
RestTemplate报错I/O error on POST request for "http://crmjob.xxx.xxx.com/removeJob": Read timed out; nested exception is java.net.SocketTimeoutException: Read timed out问题处理
讲述RestTemplate报错I/O error on POST request for "http://crmjob.xxx.xxx.com/removeJob": Read timed out; nested exception is java.net.SocketTimeoutException: Read timed out处理方案
File操作-InputStreamReader/Writer(转换器)
File操作-InputStreamReader/Writer(转换器)
55 1
|
应用服务中间件 nginx
IOException: Broken pipe
IOException: Broken pipe
157 0
|
Java 容器
Java 是否应该在HttpServletResponse.getOutputStream()/getWriter()上调用.close()?
Java 是否应该在HttpServletResponse.getOutputStream()/getWriter()上调用.close()?
164 0
OkHttp responsejava.lang.IllegalStateException: closed
OkHttp responsejava.lang.IllegalStateException: closed
|
Web App开发 Java Apache
java.lang.IllegalStateException: getOutputStream() has already been called for this response
https://stackoverflow.com/questions/33982515/handlerinterceptor-getoutputstream-has-already-been-called-for-this-response http://blog.
3810 0
|
JSON 数据格式
Could not write JSON: (was java.lang.NullPointerException); nested exception is...不出意外就是这个原因
出现这个报错百分之80都是实体类有问题,检查下实体类的get/set方法,有没有空指针 我这里是这样的,get方法忘了空判断
1815 0
|
移动开发
RN Exception: OkHttp--IllegalStateException: closed
异常 java.lang.IllegalStateException: closed at okio.RealBufferedSink.write(RealBufferedSink.
1493 0
|
前端开发 .NET 开发框架