在E-Mapreduce运行hadoop job失败,java.lang.reflect.InvocationTargetException-问答-阿里云开发者社区-阿里云

开发者社区> 问答> 正文

在E-Mapreduce运行hadoop job失败,java.lang.reflect.InvocationTargetException

2016-04-18 16:15:20 9009 1

按照文档https://help.aliyun.com/document_detail/emr/best-practice/Hadoop/Hive.html?spm=5176.docemr/trouble-shooting/oss-ram-control.6.143.tQrNyB
的sample1运行作业报错

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/apps/apache-hive-2.0.0-bin/lib/hive-jdbc-2.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/apps/apache-hive-2.0.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/apps/hbase-1.1.1/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/apps/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/opt/apps/apache-hive-2.0.0-bin/lib/hive-common-2.0.0.jar!/hive-log4j2.properties
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.aliyun.fs.oss.utils.OSSClientAgent.<init>(OSSClientAgent.java:73)
    at com.aliyun.fs.oss.nat.JetOssNativeFileSystemStore.initialize(JetOssNativeFileSystemStore.java:109)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
    at com.sun.proxy.$Proxy30.initialize(Unknown Source)
    at com.aliyun.fs.oss.nat.NativeOssFileSystem.initialize(NativeOssFileSystem.java:187)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2625)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2607)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
    at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:443)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:750)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:717)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: com.aliyun.oss.common.auth.InvalidCredentialsException: Access key id should not be null or empty.
    at com.aliyun.oss.common.auth.DefaultCredentialProvider.checkCredentials(DefaultCredentialProvider.java:63)
    at com.aliyun.oss.common.auth.DefaultCredentialProvider.<init>(DefaultCredentialProvider.java:38)
    at com.aliyun.oss.common.auth.DefaultCredentialProvider.<init>(DefaultCredentialProvider.java:34)
    at com.aliyun.oss.OSSClient.<init>(OSSClient.java:220)
    ... 28 more
Exception in thread "9842320a-d423-4a90-bcf8-5600695e37a2 main" java.io.IOException: java.lang.reflect.InvocationTargetException
    at com.aliyun.fs.oss.nat.NativeOssFileSystem.initialize(NativeOssFileSystem.java:190)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2625)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2607)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
    at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:443)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:750)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:717)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.aliyun.fs.oss.utils.OSSClientAgent.<init>(OSSClientAgent.java:73)
    at com.aliyun.fs.oss.nat.JetOssNativeFileSystemStore.initialize(JetOssNativeFileSystemStore.java:109)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
    at com.sun.proxy.$Proxy30.initialize(Unknown Source)
    at com.aliyun.fs.oss.nat.NativeOssFileSystem.initialize(NativeOssFileSystem.java:187)
    ... 14 more
Caused by: com.aliyun.oss.common.auth.InvalidCredentialsException: Access key id should not be null or empty.
    at com.aliyun.oss.common.auth.DefaultCredentialProvider.checkCredentials(DefaultCredentialProvider.java:63)
    at com.aliyun.oss.common.auth.DefaultCredentialProvider.<init>(DefaultCredentialProvider.java:38)
    at com.aliyun.oss.common.auth.DefaultCredentialProvider.<init>(DefaultCredentialProvider.java:34)
    at com.aliyun.oss.OSSClient.<init>(OSSClient.java:220)
    ... 28 more
取消 提交回答
全部回答(1)
  • 封神
    2019-07-17 18:44:07

    这个问题 ,直接可以看到错误的原因是:
    Caused by: com.aliyun.oss.common.auth.InvalidCredentialsException: Access key id should not be null or empty.

    登陆到master机器,sudo su hadoop账号下,
    运行case是没有问题的。

    用户是用e-mapreduce控制平台出的问题。
    在check过程中
    screenshot

    发现 应用参数是oss:// 应该需要写成ossref://打头的

    0 0
相关问答

41

回答

[@徐雷frank][¥20]什么是JAVA的平台无关性

大河人家 2018-10-29 23:55:20 146742浏览量 回答数 41

170

回答

惊喜翻倍:免费ECS+免费环境配置~!(ECS免费体验6个月活动3月31日结束)

豆妹 2014-10-29 17:52:21 232368浏览量 回答数 170

8

回答

OceanBase 使用动画(持续更新)

mq4096 2019-02-20 17:16:36 340544浏览量 回答数 8

13

回答

[@饭娱咖啡][¥20]我想知道 Java 关于引用那一块的知识

心意乱 2018-10-31 18:44:12 143326浏览量 回答数 13

119

回答

OSS存储服务-客户端工具

newegg11 2012-05-17 15:37:18 301126浏览量 回答数 119

22

回答

爬虫数据管理【问答合集】

我是管理员 2018-08-10 16:37:41 148558浏览量 回答数 22

24

回答

阿里云开放端口权限

xcxx 2016-07-20 15:03:33 658161浏览量 回答数 24

31

回答

[@倚贤][¥20]刚学完html/css/js的新手学习servlet、jsp需要注意哪些问题?

弗洛伊德6 2018-10-27 21:52:43 147553浏览量 回答数 31

43

回答

【精品问答集锦】Python热门问题

小六码奴 2019-05-30 15:27:34 142901浏览量 回答数 43

10

回答

[@墨玖tao][¥20]为什么流式处理框架都是 java 写成的,JVM 是不是在流和批存在着特殊优势。还有分布式资源调度,感觉Mesos 的成长速度跟不上 Yarn。这是为什么?

管理贝贝 2018-10-23 13:18:03 137443浏览量 回答数 10
+关注
封神
封神 09年加入阿里巴巴,阿里云高级技术专家、架构师;专注在大数据、分布式数据库领域,10年分布式研发经验;参与研发万台大数据Hadoop离线计算平台;负责研发多款阿里云一级大数据及数据库产品,如云Cassandra
34
文章
259
问答
问答排行榜
最热
最新
相关电子书
更多
JS零基础入门教程(上册)
立即下载
性能优化方法论
立即下载
手把手学习日志服务SLS,云启实验室实战指南
立即下载