记录安装步骤，网上有点乱
安装过程简单，主要是前面步骤做好就行
参考地址 http://blog.csdn.net/qyf_5445/article/details/42679857
包下载地址
jdk
http://www.oracle.com/technetwork/java/javase/downloads/java-archive-downloads-javase7-521261.html jdk1.7.0_79
hadoop
http://archive.cloudera.com/cdh5/cdh/5/ hadoop-2.3.0-cdh5.1.2
主机名 ip 角色
master 192.168.5.30 NameNode ResourceManager
slave1 192.168.5.31 DateNode NodeManager
slave2 192.168.5.32 DateNode NodeManager
以下操作3个节点都需要做
1.cat /etc/hosts
192.168.5.30 master
192.168.5.31 slave1
192.168.5.32 slave2
2.
useradd hadoop
passwd hadoop
3.ssh互信,su hadoop(这步在master做即可)
ssh-keygen -t rsa
ssh-copy-id -i id_rsa.pub hadoop@slave1
ssh-copy-id -i id_rsa.pub hadoop@slave2
4.
mkdir -p /data/hadoop
chown -R hadoop.hadoop /data/hadoop/
5.java环境
cat /etc/profile
export JAVA_HOME=/usr/java/jdk1.7.0_79
export JAVA_BIN=$JAVA_HOME/bin
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
export HADOOP_HOME=/data/hadoop
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$JAVA_HOME/bin:$PATH
------------------------------------------------------
以下在master上操作
6.修改/etc/hadoop/hadoop-env.sh中设JAVA_HOME。
应当使用绝对路径。
export JAVA_HOME=$JAVA_HOME //错误，不能这么改
export JAVA_HOME=/usr/java/jdk1.7.0_79 //正确，应该这么改
7.
tar xvf hadoop-2.3.0-cdh5.1.2.tar.gz -C /data/hadoop
确认目录结构是/data/hadoop/etc/hadoop
8.master上/data/hadoop/etc/hadoop
vi slaves
slave1
slave2
vi masters
master
9.修改如下几个文件，并在中间添加以下相应内容
vi core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://master:9000</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>131072</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>file:/data/hadoop/tmp</value>
</property>
</configuration>
vi hdfs-site.xml
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/data/hadoop/dfs/name</value>
</property>
<property>
<name>dfs.namenode.data.dir</name>
<value>file:/data/hadoop/dfs/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
vi yarn-site.xml
<configuration>
<property>
<name>yarn.resourcemanager.address</name>
<value>master:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>master:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>master:8088</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>
vi mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>master:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>master:19888</value>
</property>
</configuration>
10.复制hadoop到slave1,slave2
su hadoop
scp -r /data/hadoop/* hadoop@slave1:/data/hadoop/
scp -r /data/hadoop/* hadoop@slave2:/data/hadoop/
11.su hadoop,hadoop namenode -format 初始化只需要做一次
12.su hadoop,start-dfs.sh and start-yarn.sh
提示logging to /data/hadoop/logs/hadoop-hadoop-namenode-master.out
13.检测是否安装成功
jps
http://192.168.5.30:8088/cluster
14.报错
hadoop-2.3.0安装错误
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform hadoop本地库与系统版本不一致引起的错误
hadoop-2.2.0安装错误
在使用./sbin/start-dfs.sh或./sbin/start-all.sh启动时会报出这样如下警告：
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library
/usr/local/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which might have disabled
stack guard. The VM will try to fix the stack guard now.
....
Java: ssh: Could not resolve hostname Java: Name or service not known
HotSpot(TM): ssh: Could not resolve hostname HotSpot(TM): Name or service not
known
64-Bit: ssh: Could not resolve hostname 64-Bit: Name or service not known
....
这个问题的错误原因会发生在64位的操作系统上，原因是从官方下载的hadoop使用的本地库
文件(例如lib/native/libhadoop.so.1.0.0)都是基于32位编译的，运行在64位系统上就会出
现上述错误。
解决方法之一是在64位系统上重新编译hadoop，另一种方法是在hadoop-env.sh和yarn-env.sh中添加如下两行：
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
15.hadoop使用端口说明
16.ss -an
本文转自 liqius 51CTO博客，原文链接：http://blog.51cto.com/szgb17/1691814，如需转载请自行联系原作者
