官网手册：
http://hadoop.apache.org/docs/r1.0.4/cn/hdfs_design.html
主机信息
主机名 ip
hadoop1 10.0.70.242
hadoop2 10.0.70.243
hadoop3 10.0.70.230
hadoop4 10.0.70.231
一．设置主机名映射
# vim /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.70.242 hadoop1
10.0.70.243 hadoop2
10.0.70.230 hadoop3
10.0.70.231 hadoop4
并拷贝到其他3台机器。
二．配置免密码登录
# cd /root/.ssh/
# ssh-keygen -t rsa
# cat id_rsa.pub >> authorized_keys
将其他3台机器的id_rsa.pub内容追加到authorized_keys文件中，并拷贝给其他3台机器。
#ssh hadoop1（hadoop2、hadoop3、hadoop4）进行验证。
三．安装jdk
# tar zxvf jdk-8u51-linux-x64.tar.gz -C/app/zpy/
配置环境变量
# vim /etc/profile
JAVA_HOME=/app/zpy/jdk1.8.0_51
JAVA_BIN=/app/zpy/jdk1.8.0_51/bin
PATH=$PATH:$JAVA_BIN
export JAVA_HOME JAVA_BIN PATH
# . /etc/profile
# java –version 进行验证
其他3台机器相同操作。
四．安装hadoop
# cd /app/zpy/3rd
# wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.7.3/hadoop-2.7.3.tar.gz
# tar zxvf hadoop-2.7.3.tar.gz -C /app/zpy/
配置环境变量
# vim /etc/profile
HADOOP_HOME=/app/zpy/hadoop-2.7.3
HADOOP_BIN=/app/zpy/hadoop-2.7.3/bin
PATH=$PATH:$JAVA_BIN:$HADOOP_HOME:$HADOOP_BIN
export JAVA_HOME JAVA_BIN PATH HADOOP_HOMEHADOOP_BIN
# . /etc/profile
修改主要配置文件
# cd /app/zpy/hadoop-2.7.3/etc/hadoop
# vim hadoop-env.sh
添加
export JAVA_HOME=/app/zpy/jdk1.8.0_51
export HADOOP_HOME_WARN_SUPPRESS=1
# vim yarn-env.sh
更改
JAVA_HOME=/app/zpy/jdk1.8.0_51
# vim core-site.xml
内容如下
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://hadoop1:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/data/tmp</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>131702</value>
</property>
</configuration>
# vim hdfs-site.xml
内容如下
<configuration>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/data/hdfs/name/</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/data/hdfs/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>hadoop1:9001</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
</configuration>
# cp mapred-site.xml.templatemapred-site.xml
# vim mapred-site.xml
内容如下
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
# vim yarn-site.xml
内容如下
<configuration>
<!-- Site specific YARN configurationproperties -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>hadoop1:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>hadoop1:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>hadoop1:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>hadoop1:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>hadoop1:8088</value>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>3000</value>
</property>
</configuration>
# vim slaves
内容如下
hadoop1
hadoop2
hadoop3
hadoop4
创建必须的文件夹
# mkdir -p /data/tmp/
#mkdir /data/hdfs
# mkdir /data/hdfs/data
# mkdir /data/hdfs/name
五．将配置好的文件拷贝到其他主机
# scp –r hadoop-2.7.3 hadoop2:/app/zpy
# scp –r hadoop-2.7.3 hadoop3:/app/zpy
# scp –r hadoop-2.7.3 hadoop4:/app/zpy
# scp /etc/profile hadoop2:/etc
# scp /etc/profile hadoop3:/etc
# scp /etc/profile hadoop4:/etc
刷新另外3台服务器的环境变量。
六．启动
首先格式化namenode节点，注意在master节点上！
# cd /app/zpy/hadoop-2.7.3/bin/
# hadoop namenode –format
然后启动集群
# start-dfs.sh
# start-yarn.sh
在master节点上运行
# jps
看到如下结果：
4083 Jps
30084 DataNode
30261 SecondaryNameNode
30550 NodeManager
29945 NameNode
30430 ResourceManager
切换到slave节点，运行
# jps
看到如下结果：
18561 NodeManager
4362 Jps
18443 DataNode
浏览器中打开http://10.0.70.242:50070/dfshealth.html#tab-overview可看见相应信息
七．运行测试用例
切换到hadoop根目录
执行如下命令解除ｈａｄｏｏｐ安全模式：
# ./bin/hadoop dfsadmin -safemode leave
再执行：
# ./bin/hadoop jarshare/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar pi 1 1
显示如下结果：
Number of Maps = 1
Samples per Map = 1
Wrote input for Map #0
Starting Job
．．．．．．
Job Finished in 15.972 seconds
Estimated value of Pi is 4.00000000000000000000
证明成功，至此完成！
本文转自 周新宇1991 51CTO博客，原文链接：http://blog.51cto.com/zhouxinyu1991/1870183，如需转载请自行联系原作者
