开发者社区> 问答> 正文

hadoop伪分布式运行wordcount出错

在服务器上配置了一下伪分布式的hadoop环境,执行它自带的mapreduce都出错了,知道你们很厉害,特意问一下。新手并不太会呢,帮看看好不。
有一些节点自己就莫名其妙的死了,我晚上推出集群都不关,第二天在连接ssh,集群自己就关了是为啥
和环境有关吗

[gaojunyu@gaojunyu hadoop-2.6.0]$ bin/hadoop jar /home/gaojunyu/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount /input/wordcount /output/wordcount
16/11/22 09:40:44 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/11/22 09:40:46 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
16/11/22 09:40:47 INFO input.FileInputFormat: Total input paths to process : 1
16/11/22 09:40:47 INFO mapreduce.JobSubmitter: number of splits:1
16/11/22 09:40:48 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1479775086295_0002
16/11/22 09:40:48 INFO impl.YarnClientImpl: Submitted application application_1479775086295_0002
16/11/22 09:40:48 INFO mapreduce.Job: The url to track the job: http://gaojunyu:8088/proxy/application_1479775086295_0002/
16/11/22 09:40:48 INFO mapreduce.Job: Running job: job_1479775086295_0002
16/11/22 09:40:58 INFO mapreduce.Job: Job job_1479775086295_0002 running in uber mode : false
16/11/22 09:40:58 INFO mapreduce.Job: map 0% reduce 0%
16/11/22 09:41:06 INFO mapreduce.Job: Task Id : attempt_1479775086295_0002_m_000000_0, Status : FAILED
Exception from container-launch.
Container id: container_1479775086295_0002_01_000002
Exit code: 1
Stack trace: ExitCodeException exitCode=1:

at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Container exited with a non-zero exit code 1

16/11/22 09:41:15 INFO mapreduce.Job: map 100% reduce 0%
16/11/22 10:01:15 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
16/11/22 10:01:16 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
16/11/22 10:01:17 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
16/11/22 10:01:18 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
16/11/22 10:01:19 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
机密信息 :
2016-11-22 10:14:59

展开
收起
a_宇 2016-11-22 10:57:31 3391 0
2 条回答
写回答
取消 提交回答
  • resourcemanager挂了,结合后续ssh登陆不上,可能是sshd发生oom被杀,推测之前各种jvm也发生了oom,很有可能是参数配置不合理,如xmx的值过大导致

    2019-07-17 20:28:06
    赞同 展开评论 打赏
  • 估计是nodemanager进程挂了,需要看一下对应的日志

    2019-07-17 20:28:06
    赞同 展开评论 打赏
问答排行榜
最热
最新

相关电子书

更多
《构建Hadoop生态批流一体的实时数仓》 立即下载
零基础实现hadoop 迁移 MaxCompute 之 数据 立即下载
CIO 指南:如何在SAP软件架构中使用Hadoop 立即下载

相关实验场景

更多