1 前言
Cloudera是基于Apache原生的Hadoop组件进行重新封装和加强，Cloudera可以简化Hadoop组件的部署。
更新版本的配置请参阅（CentOS 7）：
http://cmdschool.blog.51cto.com/2420395/1916322
2 理论基础
2.1 部署软件架构
1）Oracle JDK
2）Cloudera Manager Server and Agent packages
3）Supporting database software
4）CDH and managed service software
2.2 部署步骤和安装方法
2.2.1 安装方法
A）Cloudera Manager安装程序安装（容易）
B）yum源方式安装（中等）
C）源代码安装（难）
注：本教程使用方法B
2.2.2 部署步骤
1）安装JDK
2）安装并配置数据库
3）安装Cloudera管理服务端
4）安装Cloudera管理代理端
5）安装CDH管理服务软件
6）创建、启动和配置CDH并管理服务
2.3 Cloudera Manager端的相关文件
|
1
|
rpm -ql cloudera-manager-server
|
显示如下：
|
1
2
3
4
5
6
7
8
9
10
|
/etc/cloudera-scm-server
/etc/cloudera-scm-server/db
.properties
/etc/cloudera-scm-server/log4j
.properties
/etc/default/cloudera-scm-server
/etc/rc
.d
/init
.d
/cloudera-scm-server
/opt/cloudera/csd
/opt/cloudera/parcel-repo
/usr/sbin/cmf-server
/var/log/cloudera-scm-server
/var/run/cloudera-scm-server
|
文件与目录功能如下：
1）其中/etc/的2-4行为Cloudera Manager服务端配置文件
2）/opt/cloudera/parcel-repo为下载安装包存放目录
3 实践部分
3.1 环境信息
3.1.1 系统信息
OS = CentOS 6.6 x86_64
注：系统请使用最小化安装，否则可能Sqoop服务可能无法启动
3.1.2 主机信息
Cloudera Manager:
ip address=10.168.0.120
hostname=cdm-m.cmdschool.org
Cloudera Host1:
ip address=10.168.0.121
hostname=cdm-h1.cmdschool.org
Cloudera Host2:
ip address=10.168.0.122
hostname=cdm-h2.cmdschool.org
Cloudera Host3:
ip address=10.168.0.123
hostname=cdm-h3.cmdschool.org
Cloudera Host4:
ip address=10.168.0.124
hostname=cdm-h4.cmdschool.org
3.2 运行环境配置
In Cloudera Manager & Cloudera Host[1-4]
3.2.1 关闭selinux
|
1
|
getenforce
|
如果显示如下：
|
1
|
Enforcing
|
则执行：
|
1
2
|
setenforce 0
sed
-i
's/SELINUX=enforcing/SELINUX=disabled/g'
/etc/selinux/config
|
3.2.2 配置hosts
vim编辑/etc/hosts
|
1
2
3
4
5
|
10.168.0.120 cdm-m.cmdschool.org
10.168.0.121 cdm-h1.cmdschool.org
10.168.0.122 cdm-h2.cmdschool.org
10.168.0.123 cdm-h3.cmdschool.org
10.168.0.124 cdm-h4.cmdschool.org
|
3.2.3 检查主机名称
|
1
|
hostname
|
注：主机名称与上面不一致会影响服务的启动
3.2.4 配置sudo(单用户模式适用，可选)
|
1
|
visudo
|
增加如下组
|
1
|
%cloudera-scm ALL=(ALL) NOPASSWD: ALL
|
确认包含如下行：
|
1
|
Defaults secure_path =
/sbin
:
/bin
:
/usr/sbin
:
/usr/bin
|
vim编辑/etc/pam.d/su，确保包含如下行
|
1
|
session required pam_limits.so
|
3.2.5 关闭防火墙并设置开机不启动
|
1
2
|
/etc/init
.d
/iptables
stop
chkconfig iptables off
|
3.2.6 优化虚拟内存需求率
1)检查虚拟内存需求率
|
1
|
cat
/proc/sys/vm/swappiness
|
显示如下：
|
1
|
|
2)临时降低虚拟内存需求率
|
1
|
sysctl vm.swappiness=0
|
3)永久降低虚拟内存需求率
vim编辑/etc/sysctl.conf
|
1
2
|
kernel.shmall = 4294967296
vm.swappiness = 0
|
并运行如下命令使生效
|
1
|
sysctl -p
|
3.2.7 解决透明大页面问题
1)检查透明大页面问题
|
1
|
cat
/sys/kernel/mm/transparent_hugepage/defrag
|
如果显示为：
|
1
|
[always] madvise never
|
2)临时关闭透明大页面问题
|
1
|
echo
never >
/sys/kernel/mm/transparent_hugepage/defrag
|
确认配置生效：
|
1
|
cat
/sys/kernel/mm/transparent_hugepage/defrag
|
应该显示为：
|
1
|
always madvise [never]
|
3)配置开机自动生效
vim编辑/etc/rc.local,加入如下内容
|
1
|
echo
never >
/sys/kernel/mm/transparent_hugepage/defrag
|
3.3 yum源的安装配置
3.3.1 公共yum源配置
In Cloudera Manager & Cloudera Host[1-4]
1)配置yum源
下载默认yum源
|
1
|
wget -P
/etc/yum
.repos.d/ https:
//archive
.cloudera.com
/cm5/redhat/6/x86_64/cm/cloudera-manager
.repo
|
修改为指定版本yum源
vim编辑/etc/yum.repos.d/cloudera-manager.repo修改如下参数：
|
1
|
baseurl=https:
//archive
.cloudera.com
/cm5/redhat/6/x86_64/cm/5
.6.0/
|
2)安装配置工具
|
1
|
yum
install
-y vim wget openssh-clients
|
3)安装jdk
|
1
|
yum
install
-y oracle-j2sdk1.7
|
4)安装python
|
1
|
yum
install
-y python
|
5)安装ntpd
|
1
|
yum
install
-y ntp
|
3.3.2 Cloudera Manager端yum源配置
In Cloudera Manager
1)安装Cloudera Manager包
|
1
|
yum
install
-y cloudera-manager-daemons cloudera-manager-server
|
2)安装mysql
|
1
|
yum
install
-y mysql-server mysql-devel mysql
|
3.3.3 Cloudera Manager Agent端yum源配置
In Cloudera Host[1-4]
安装Cloudera Manager Agent包
|
1
|
yum
install
-y cloudera-manager-agent cloudera-manager-daemons
|
3.4 依赖于yum源的环境配置
3.4.1 配置jdk变量环境
In Cloudera Manager & Cloudera Host[1-4]
1)vim编辑/etc/profile，末尾加入如下内容
|
1
2
3
4
|
export
JAVA_HOME=
/usr/java/jdk1
.7.0_67-cloudera
export
JRE_HOME=${JAVA_HOME}
/jre
export
CLASSPATH=.:${JAVA_HOME}
/lib
:${JRE_HOME}
/lib
export
PATH=${JAVA_HOME}
/bin
:$PATH
|
2)导入java环境变量
|
1
|
source
/etc/profile
|
3)测试jdk的配置
|
1
|
java -version
|
3.4.2 权限检查(单用户模式适用，可选)
In Cloudera Manager & Cloudera Host[1-4]
检查以下目录cloudera-scm用户具有完全的权限
检查当前目录权限:
|
1
|
ls
-ld
/opt/cloudera/
|
显示如下
|
1
|
drwxr-xr-x. 4 cloudera-scm cloudera-scm 4096 May 23 13:51
/opt/cloudera/
|
检查子目录权限:
|
1
|
ls
-lR
/opt/cloudera/
|
显示如下
|
1
2
3
4
5
6
7
8
9
10
|
/opt/cloudera/
:
total 8
drwxr-xr-x. 2 cloudera-scm cloudera-scm 4096 Feb 12 11:28 csd
drwxr-xr-x. 2 cloudera-scm cloudera-scm 4096 Feb 12 11:28 parcel-repo
/opt/cloudera/csd
:
total 0
/opt/cloudera/parcel-repo
:
total 0
|
同样，检查服务器或客户端目录权限
|
1
2
3
4
|
ls
-ld
/var/log/cloudera-scm-server/
ls
-lR
/var/log/cloudera-scm-server/
ls
-ld
/var/lib/cloudera-scm-agent/
ls
-lR
/var/lib/cloudera-scm-agent/
|
3.4.3 检查线程限制配置
In Cloudera Manager & Cloudera Host[1-4]
|
1
|
cat
/etc/security/limits
.d
/cloudera-scm
.conf
|
显示如下：
|
1
2
3
4
5
6
7
8
|
#
# (c) Copyright 2014 Cloudera, Inc.
#
cloudera-scm soft nofile 32768
cloudera-scm soft nproc 65536
cloudera-scm hard nofile 1048576
cloudera-scm hard nproc unlimited
cloudera-scm hard memlock unlimited
|
3.4.4 Cloudera Manager端配置
In Cloudera Manager
1)临时校对时间
|
1
|
ntpdate 0.centos.pool.ntp.org
|
2)启动并配置ntpd服务自动启动
|
1
2
|
/etc/init
.d
/ntpd
start
chkconfig ntpd on
|
3.4.5 Cloudera Manager Agen端配置
In Cloudera Host[1-4]
1)临时校对时间
|
1
|
ntpdate 10.168.0.120
|
2)vim编辑/etc/ntp.conf
注释掉外网时间服务器并增加内网时间服务器地址
|
1
2
3
4
5
|
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst
server 10.168.0.120 iburst
|
3)启动并配置ntpd服务自动启动
|
1
2
|
/etc/init
.d
/ntpd
start
chkconfig ntpd on
|
3.4.7 安装MySQL JDBC Driver
In Cloudera Manager & Cloudera Host[1-4]
|
1
2
3
4
|
wget http:
//dev
.mysql.com
/get/Downloads/Connector-J/mysql-connector-java-5
.1.39.
tar
.gz
tar
zxvf mysql-connector-java-5.1.39.
tar
.gz
mkdir
/usr/share/java/
cp
mysql-connector-java-5.1.39
/mysql-connector-java-5
.1.39-bin.jar
/usr/share/java/mysql-connector-java
.jar
|
3.4.8 配置公钥认证
In Cloudera Manager
In Cloudera Manager：
|
1
|
ssh
-keygen -t rsa
|
注：以上一路回车
In Cloudera Manager Agen：
|
1
2
3
4
5
|
ssh
-copy-
id
-i ~/.
ssh
/id_rsa
.pub root@10.168.0.120
ssh
-copy-
id
-i ~/.
ssh
/id_rsa
.pub root@10.168.0.121
ssh
-copy-
id
-i ~/.
ssh
/id_rsa
.pub root@10.168.0.122
ssh
-copy-
id
-i ~/.
ssh
/id_rsa
.pub root@10.168.0.123
ssh
-copy-
id
-i ~/.
ssh
/id_rsa
.pub root@10.168.0.124
|
In Cloudera Manager：
|
1
2
3
4
5
|
ssh
10.168.0.120
ssh
10.168.0.121
ssh
10.168.0.122
ssh
10.168.0.123
ssh
10.168.0.124
|
注：以上如果无需密码登记即成功
3.5 Cloudera Manager安装配置
In Cloudera Manager
3.5.1 修改mysql参数
1)关闭数据库
|
1
|
/etc/init
.d
/mysqld
stop
|
2)备份ib_logfile文件
|
1
2
3
|
mkdir
/var/lib/backup
cd
/var/lib/mysql/
mv
ib_logfile*
/var/lib/backup/
|
3)vim编辑/etc/my.cnf
加入如下参数
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
|
[mysqld]
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
# symbolic-links = 0
key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1
max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M
#log_bin should be on a disk with enough free space. Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your system
#and chown the specified folder to the mysql user.
log_bin=
/var/lib/mysql/mysql_binary_log
# For MySQL version 5.1.8 or later. Comment out binlog_format for older versions.
binlog_format = mixed
read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M
# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M
[mysqld_safe]
log-error=
/var/log/mysqld
.log
pid-
file
=
/var/run/mysqld/mysqld
.pid
sql_mode=STRICT_ALL_TABLES
|
3.5.2 启动并设置开机自动启动
|
1
2
|
/etc/init
.d
/mysqld
start
chkconfig mysqld on
|
3.5.3 初始化数据库
|
1
|
mysql_secure_installation
|
向导如下：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
[...]
Enter current password
for
root (enter
for
none):
OK, successfully used password, moving on...
[...]
Set root password? [Y
/n
] y
New password:
Re-enter new password:
Remove anonymous
users
? [Y
/n
] y
[...]
Disallow root login remotely? [Y
/n
] n
[...]
Remove
test
database and access to it [Y
/n
] y
[...]
Reload privilege tables now? [Y
/n
] y
All
done
!
|
3.5.4 准备scm库
1)方法一
数据库配置：
|
1
2
3
4
|
mysql -uroot -p
create database scm default character
set
utf8;
grant all privileges on *.* to scm@
'cdm-m.cmdschool.org'
identified by
'scm'
;
flush privileges;
|
vim编辑/etc/cloudera-scm-server/db.properties修改如下参数：
|
1
2
3
4
5
|
com.cloudera.cmf.db.
type
=mysql
com.cloudera.cmf.db.host=cdm-m.cmdschool.org
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.password=scm
|
2)方法二(官方建议)
授权temp权限：
|
1
2
3
|
mysql -uroot -p
grant all privileges on *.* to
'temp'
@
'%'
identified by
'temp'
with grant option;
flush privileges;
|
生成配置文件：
|
1
|
/usr/share/cmf/schema/scm_prepare_database
.sh mysql -h cdm-m.cmdschool.org -utemp -ptemp --scm-host cdm-m.cmdschool.org scm scm scm
|
显示如下：
|
1
2
3
4
5
6
|
JAVA_HOME=
/usr/java/jdk1
.7.0_67-cloudera
Verifying that we can write to
/etc/cloudera-scm-server
Creating SCM configuration
file
in
/etc/cloudera-scm-server
Executing:
/usr/java/jdk1
.7.0_67-cloudera
/bin/java
-
cp
/usr/share/java/mysql-connector-java
.jar:
/usr/share/java/oracle-connector-java
.jar:
/usr/share/cmf/schema/
..
/lib/
* com.cloudera.enterprise.dbutil.DbCommandExecutor
/etc/cloudera-scm-server/db
.properties com.cloudera.cmf.db.
[main] DbCommandExecutor INFO Successfully connected to database.
All
done
, your SCM database is configured correctly!
|
确认生成的结果：
|
1
|
cat
/etc/cloudera-scm-server/db
.properties
|
显示如下：
|
1
2
3
4
5
6
7
8
9
10
|
# Auto-generated by scm_prepare_database.sh on Tue May 24 19:08:19 CST 2016
#
# For information describing how to configure the Cloudera Manager Server
# to connect to databases, see the "Cloudera Manager Installation Guide."
#
com.cloudera.cmf.db.
type
=mysql
com.cloudera.cmf.db.host=cdm-m.cmdschool.org
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.password=scm
|
确认库访问权限:
|
1
2
|
mysql -uroot -p
show grants
for
scm@
'cdm-m.cmdschool.org'
;
|
显示如下：
|
1
2
3
4
5
6
7
|
+----------------------------------------------------------------------------------------------------------------------+
| Grants
for
scm@cdm-m.cmdschool.org |
+----------------------------------------------------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO
'scm'
@
'cdm-m.cmdschool.org'
IDENTIFIED BY PASSWORD
'*45E6E3C68BDF1AC7EBB5C5A3BCBD5E9437B293BE'
|
| GRANT ALL PRIVILEGES ON `scm`.* TO
'scm'
@
'cdm-m.cmdschool.org'
|
+----------------------------------------------------------------------------------------------------------------------+
2 rows
in
set
(0.00 sec)
|
清理数据库用户授权：
|
1
2
|
drop user
'temp'
@
'%'
;
flush privileges;
|
3.5.5 创建附加数据库(可选)
1)附加数据库列表
| Role | Database | User | Password |
| Activity Monitor | amon | amon | amon_password |
| Reports Manager | rman | rman | rman_password |
| Hive Metastore Server | metastore | hive | hive_password |
| Sentry Server | sentry | sentry | sentry_password |
| Cloudera Navigator Audit Server | nav | nav | nav_password |
Cloudera Navigator Metadata Server |
navms | navms | navms_password |
2)创建数据库并配置管理账号密码
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
mysql -uroot -p
create database amon default character
set
utf8;
grant all privileges on amon.* to
'amon'
@
'%'
identified by
'amon_password'
;
create database rman default character
set
utf8;
grant all privileges on rman.* to
'rman'
@
'%'
identified by
'rman_password'
;
create database metastore default character
set
utf8;
grant all privileges on metastore.* to
'hive'
@
'%'
identified by
'hive_password'
;
create database sentry default character
set
utf8;
grant all privileges on sentry.* to
'sentry'
@
'%'
identified by
'sentry_password'
;
create database nav default character
set
utf8;
grant all privileges on nav.* to
'nav'
@
'%'
identified by
'nav_password'
;
create database navms default character
set
utf8;
grant all privileges on navms.* to
'navms'
@
'%'
identified by
'navms_password'
;
flush privileges;
|
3.5.6 配置Oozie库(可选)
1)数据库权限配置
|
1
2
3
4
5
|
mysql -uroot -p
create database oozie default character
set
utf8;
grant all privileges on oozie.* to
'oozie'
@
'localhost'
identified by
'oozie'
;
grant all privileges on oozie.* to
'oozie'
@
'%'
identified by
'oozie'
;
flush privileges;
|
2)配置Oozie库所需软连接
|
1
2
|
cd
/opt/cloudera/parcels/CDH/lib/oozie/lib/
ln
-s
/usr/share/java/mysql-connector-java
.jar mysql-connector-java.jar
|
3.5.7 启动服务并配置开机启动
|
1
2
|
/etc/init
.d
/cloudera-scm-server
start
chkconfig cloudera-scm-server on
|
3.5.8 故障排除
|
1
|
tail
-f
/var/log/cloudera-scm-server/cloudera-scm-server
.out
|
3.6 Cloudera Manager Agent安装
In Cloudera Host[1-4]
3.6.1 创建压缩包存放目录
|
1
2
|
mkdir
-p
/opt/cloudera/parcels
chown
cloudera-scm:cloudera-scm
/opt/cloudera/parcels
|
3.6.2 指定管理服务器和指定包存放目录
vim编辑/etc/cloudera-scm-agent/config.ini确保参数如下并启用：
|
1
2
3
|
server_host=cdm-m.cmdschool.org
server_port=7182
parcel_dir=
/opt/cloudera/parcels
|
3.6.3 指定运行单用户模式的用户名(仅用于单用户模式，不配置)
vim编辑/etc/default/cloudera-scm-agent并取消以下行的注释
|
1
|
USER=
"cloudera-scm"
|
3.6.4 启动服务并配置服务器开机启动
|
1
2
|
/etc/init
.d
/cloudera-scm-agent
start
chkconfig cloudera-scm-agent on
|
3.6.5 故障排除
如下命令监控启动服务的错误输出
|
1
|
tail
-f
/var/log/cloudera-scm-agent/cloudera-scm-agent
.out
|
3.7 登陆配置
In Cloudera WEB Manager
界面配置部分本章节省略……
更新版本的配置请参阅（CentOS 7）：
