在E-Mapreduce运行hadoop job失败,java.lang.reflect.InvocationTargetException-问答-阿里云开发者社区-阿里云

开发者社区> 问答> 正文

在E-Mapreduce运行hadoop job失败,java.lang.reflect.InvocationTargetException

封神 2016-04-18 16:15:20 5724

按照文档https://help.aliyun.com/document_detail/emr/best-practice/Hadoop/Hive.html?spm=5176.docemr/trouble-shooting/oss-ram-control.6.143.tQrNyB
的sample1运行作业报错

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/apps/apache-hive-2.0.0-bin/lib/hive-jdbc-2.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/apps/apache-hive-2.0.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/apps/hbase-1.1.1/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/apps/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/opt/apps/apache-hive-2.0.0-bin/lib/hive-common-2.0.0.jar!/hive-log4j2.properties
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.aliyun.fs.oss.utils.OSSClientAgent.<init>(OSSClientAgent.java:73)
    at com.aliyun.fs.oss.nat.JetOssNativeFileSystemStore.initialize(JetOssNativeFileSystemStore.java:109)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
    at com.sun.proxy.$Proxy30.initialize(Unknown Source)
    at com.aliyun.fs.oss.nat.NativeOssFileSystem.initialize(NativeOssFileSystem.java:187)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2625)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2607)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
    at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:443)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:750)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:717)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: com.aliyun.oss.common.auth.InvalidCredentialsException: Access key id should not be null or empty.
    at com.aliyun.oss.common.auth.DefaultCredentialProvider.checkCredentials(DefaultCredentialProvider.java:63)
    at com.aliyun.oss.common.auth.DefaultCredentialProvider.<init>(DefaultCredentialProvider.java:38)
    at com.aliyun.oss.common.auth.DefaultCredentialProvider.<init>(DefaultCredentialProvider.java:34)
    at com.aliyun.oss.OSSClient.<init>(OSSClient.java:220)
    ... 28 more
Exception in thread "9842320a-d423-4a90-bcf8-5600695e37a2 main" java.io.IOException: java.lang.reflect.InvocationTargetException
    at com.aliyun.fs.oss.nat.NativeOssFileSystem.initialize(NativeOssFileSystem.java:190)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2625)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2607)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
    at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:443)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:750)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:717)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.aliyun.fs.oss.utils.OSSClientAgent.<init>(OSSClientAgent.java:73)
    at com.aliyun.fs.oss.nat.JetOssNativeFileSystemStore.initialize(JetOssNativeFileSystemStore.java:109)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
    at com.sun.proxy.$Proxy30.initialize(Unknown Source)
    at com.aliyun.fs.oss.nat.NativeOssFileSystem.initialize(NativeOssFileSystem.java:187)
    ... 14 more
Caused by: com.aliyun.oss.common.auth.InvalidCredentialsException: Access key id should not be null or empty.
    at com.aliyun.oss.common.auth.DefaultCredentialProvider.checkCredentials(DefaultCredentialProvider.java:63)
    at com.aliyun.oss.common.auth.DefaultCredentialProvider.<init>(DefaultCredentialProvider.java:38)
    at com.aliyun.oss.common.auth.DefaultCredentialProvider.<init>(DefaultCredentialProvider.java:34)
    at com.aliyun.oss.OSSClient.<init>(OSSClient.java:220)
    ... 28 more
分布式计算 Java Hadoop
分享到
取消 提交回答
全部回答(1)
  • 封神
    2019-07-17 18:44:07

    这个问题 ,直接可以看到错误的原因是:
    Caused by: com.aliyun.oss.common.auth.InvalidCredentialsException: Access key id should not be null or empty.

    登陆到master机器,sudo su hadoop账号下,
    运行case是没有问题的。

    用户是用e-mapreduce控制平台出的问题。
    在check过程中
    screenshot

    发现 应用参数是oss:// 应该需要写成ossref://打头的

    0 0
+ 订阅

大数据计算实践乐园,近距离学习前沿技术

推荐文章
相似问题