hadoop伪分布式运行wordcount出错-问答-阿里云开发者社区-阿里云

开发者社区> 问答> 正文
阿里云
为了无法计算的价值
打开APP
阿里云APP内打开

hadoop伪分布式运行wordcount出错

2016-11-22 10:57:31 2990 2

在服务器上配置了一下伪分布式的hadoop环境,执行它自带的mapreduce都出错了,知道你们很厉害,特意问一下。新手并不太会呢,帮看看好不。
有一些节点自己就莫名其妙的死了,我晚上推出集群都不关,第二天在连接ssh,集群自己就关了是为啥
和环境有关吗

[gaojunyu@gaojunyu hadoop-2.6.0]$ bin/hadoop jar /home/gaojunyu/app/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount /input/wordcount /output/wordcount
16/11/22 09:40:44 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/11/22 09:40:46 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
16/11/22 09:40:47 INFO input.FileInputFormat: Total input paths to process : 1
16/11/22 09:40:47 INFO mapreduce.JobSubmitter: number of splits:1
16/11/22 09:40:48 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1479775086295_0002
16/11/22 09:40:48 INFO impl.YarnClientImpl: Submitted application application_1479775086295_0002
16/11/22 09:40:48 INFO mapreduce.Job: The url to track the job: http://gaojunyu:8088/proxy/application_1479775086295_0002/
16/11/22 09:40:48 INFO mapreduce.Job: Running job: job_1479775086295_0002
16/11/22 09:40:58 INFO mapreduce.Job: Job job_1479775086295_0002 running in uber mode : false
16/11/22 09:40:58 INFO mapreduce.Job: map 0% reduce 0%
16/11/22 09:41:06 INFO mapreduce.Job: Task Id : attempt_1479775086295_0002_m_000000_0, Status : FAILED
Exception from container-launch.
Container id: container_1479775086295_0002_01_000002
Exit code: 1
Stack trace: ExitCodeException exitCode=1:

at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Container exited with a non-zero exit code 1

16/11/22 09:41:15 INFO mapreduce.Job: map 100% reduce 0%
16/11/22 10:01:15 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
16/11/22 10:01:16 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
16/11/22 10:01:17 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
16/11/22 10:01:18 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
16/11/22 10:01:19 INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
机密信息 :
2016-11-22 10:14:59

取消 提交回答
全部回答(2)
  • upooo
    2019-07-17 20:28:06

    resourcemanager挂了,结合后续ssh登陆不上,可能是sshd发生oom被杀,推测之前各种jvm也发生了oom,很有可能是参数配置不合理,如xmx的值过大导致

    0 0
  • 大数据连
    2019-07-17 20:28:06

    估计是nodemanager进程挂了,需要看一下对应的日志

    0 0
添加回答
相关问答

1

回答

Hadoop作为开源分布式系统基础架构,拥有哪2个核心组件?

2022-01-13 18:04:15 482浏览量 回答数 1

1

回答

hadoop 的四大组件分别是什么呢?

2021-12-12 21:29:26 514浏览量 回答数 1

1

回答

新建 hadoop 用户的命令代码是什么?

2021-12-05 13:35:15 272浏览量 回答数 1

1

回答

Hadoop中移除和删除文件夹的命令是什么呢?

2021-12-05 13:47:01 123浏览量 回答数 1

1

回答

Hadoop中的分组用什么来实现?

2021-12-05 11:45:33 117浏览量 回答数 1

1

回答

Hadoop前期准备工作是什么?

2021-12-06 08:43:43 124浏览量 回答数 1

1

回答

hadoop的三大发行版是什么呢?

2021-12-04 17:37:04 156浏览量 回答数 1

1

回答

hadoop的集群是基于什么呢?

2021-12-04 13:56:43 125浏览量 回答数 1

1

回答

Hadoop完全分布式的运行需要进行什么操作?

2021-12-03 22:18:54 82浏览量 回答数 1

1

回答

Hadoop-2.7.1运行wordcount报错(mapreduce已经执行完?报错

2020-06-12 15:27:04 398浏览量 回答数 1
+关注
2
文章
3
问答
问答排行榜
最热
最新
相关电子书
更多
低代码开发师(初级)实战教程
立即下载
阿里巴巴DevOps 最佳实践手册
立即下载
冬季实战营第三期:MySQL数据库进阶实战
立即下载