Ubuntu14.04下完美安装cloudermanage多种方式(图文详解)(博主推荐)

本文涉及的产品
云数据库 RDS MySQL,集群系列 2核4GB
推荐场景:
搭建个人博客
RDS MySQL Serverless 基础系列,0.5-2RCU 50GB
云数据库 RDS PostgreSQL,集群系列 2核4GB
简介:

说在前面的话

  我的机器是总共4台,分别为ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4。

 

  ClouderaManager官网

https://www.cloudera.com/documentation/manager/5-0-x/Cloudera-Manager-Installation-Guide/cm5ig_install_path_B.html

 

 

 

 

  在ubuntucmbigdata1上执行了

sudo apt-get install cloudera-manager-daemons cloudera-manager-server 


sudo apt-get install cloudera-manager-daemons  cloudera-manager-agent 

  即,ubuntucmbigdata1是既做server,又做agent。

 

 

  在ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4上执行了

sudo apt-get install cloudera-manager-daemons  cloudera-manager-agent 

  即,ubuntucmbigdata2、3、和4是只做agent。

 

 

  在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上

新建文件sudo vi /etc/apt/sources.list.d/cloudera-manager.list,复制以下内容到cloudera.list里

# Packages for Cloudera Manager, Version 5, on Ubuntu 14.04 x86_64
deb [arch=amd64] http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib
deb-src http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib

 

  并且在在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上

  向apt添加cloudera源公钥,在所有节点执行如下命令:

复制代码
curl -s http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key | sudo apt-key add -

然后,再

sudo apt-get update

复制代码

 

 

 

 

 

方式1、将CDH5的软件源添加到Apt中,通过在线安装。

参考

Ubuntu14.04用apt在线/离线安装CDH5.1.2[Apache Hadoop 2.3.0]

 https://www.zybuluo.com/ncepuwanghui/note/474966

 配置Cloudera软件仓库

Cloudera官方文档提供了两种管理仓库策略: 
• Standard Cloudera repositories 
• Internally hosted repositories

本文采用Standard Cloudera repositories,步骤如下: 
① 在ubuntucmbigdata1节点下载cloudera.list

wget https://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/cloudera.list?_ga=1.37493987.2113753785.1471598714

 

② 在ubuntucmbigdata1节点新建文件sudo vi /etc/apt/sources.list.d/cloudera-manager.list,复制cloudera.list的内容

# Packages for Cloudera Manager, Version 5, on Ubuntu 14.04 x86_64
deb [arch=amd64] http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib
deb-src http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib

③ 将cloudera-manager.list分发到所有slave节点:

scp /etc/apt/sources.list.d/cloudera-manager.list bigdata@ubuntucmbigdata2:/etc/apt/sources.list.d/cloudera-manager.list
scp /etc/apt/sources.list.d/cloudera-manager.list bigdata@ubuntucmbigdata3:/etc/apt/sources.list.d/cloudera-manager.list
scp /etc/apt/sources.list.d/cloudera-manager.list bigdata@ubuntucmbigdata4:/etc/apt/sources.list.d/cloudera-manager.list

 

  1. ④ 向apt添加cloudera源公钥,在所有节点执行如下命令:
curl -s http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key | sudo apt-key add -

 

  1. ⑤ 在所有节点,更新源
sudo apt-get update

 

 

若更新出错,提示如下:

  1. W: GPG error: http://archive.cloudera.com trusty-cm5 InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 327574EE02A818DD

可执行如下命令,添加cloudera源的公钥,重新更新源:

    1. curl -s http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key | sudo apt-key add -

   

 

  

 

安装Cloudera Manager Server

    • 安装Cloudera Manager Server软件包 
      maubuntucmbigdata1ster节点上执行:

      1. sudo apt-get install cloudera-manager-daemons cloudera-manager-server
    • 创建Cloudera Manager Server数据库scm

    • 复制代码
      mysql -u root -p
      create database scm DEFAULT CHARACTER SET utf8;
      grant all on scm.* TO 'scm'@'%' IDENTIFIED BY 'scm_password';
      grant all on *.* TO 'scm'@'ubuntucmbigdata1' IDENTIFIED BY 'scm_password';
      grant all on scm.* TO 'scm'@'localhost' IDENTIFIED BY 'scm_password';
      flush privileges;
      复制代码

       

    • 同时,关于这里的mysql配置,还要加上,我这篇博客里的mysql内容

      Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

      • 复制代码
        mysql> create database hive DEFAULT CHARACTER SET utf8;
        Ignoring query to other database
        mysql> grant all on hive.* TO 'hive'@'localhost' IDENTIFIED BY 'hive_password';
        Ignoring query to other database
        mysql> grant all on *.* TO 'hive'@'ubuntucmbigdata1' IDENTIFIED BY 'hive_password';
        Ignoring query to other database
        mysql> grant all on hive.* TO 'hive'@'%' IDENTIFIED BY 'hive_password';  
        Ignoring query to other database
        mysql> create database oozie DEFAULT CHARACTER SET utf8;
        Ignoring query to other database
        mysql> grant all on oozie.* TO 'oozie'@'localhost' IDENTIFIED BY 'oozie_password';
        Ignoring query to other database
        mysql> grant all on *.* TO 'oozie'@'ubuntucmbigdata1' IDENTIFIED BY 'oozie_password';
        Ignoring query to other database
        mysql> grant all on oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie_password';
        Ignoring query to other database
        mysql> create database amon DEFAULT CHARACTER SET utf8;
        Ignoring query to other database
        mysql> grant all on amon.* TO 'amon'@'localhost' IDENTIFIED BY 'amon_password';
        Ignoring query to other database
        mysql> grant all on *.* TO 'amon'@'ubuntucmbigdata1' IDENTIFIED BY 'amon_password';
        Ignoring query to other database
        mysql> grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon_password';
        Ignoring query to other database
        mysql> create database smon DEFAULT CHARACTER SET utf8;
        Ignoring query to other database
        mysql> grant all on smon.* TO 'smon'@'localhost' IDENTIFIED BY 'smon_password';
        Ignoring query to other database
        mysql> grant all on *.* TO 'smon'@'ubuntucmbigdata1' IDENTIFIED BY 'smon_password';
        Ignoring query to other database
        mysql> grant all on smon.* TO 'smon'@'%' IDENTIFIED BY 'smon_password';
        Ignoring query to other database
        mysql> create database rman DEFAULT CHARACTER SET utf8;
        Ignoring query to other database
        mysql> grant all on rman.* TO 'rman'@'localhost' IDENTIFIED BY 'rman_password';
        Ignoring query to other database
        mysql> grant all on *.* TO 'rman'@'ubuntucmbigdata1' IDENTIFIED BY 'rman_password';
        Ignoring query to other database
        mysql> grant all on rman.* TO 'rman'@'%' IDENTIFIED BY 'rman_password';
        Ignoring query to other database
        mysql> create database hmon DEFAULT CHARACTER SET utf8;
        Ignoring query to other database
        mysql> grant all on hmon.* TO 'hmon'@'localhost' IDENTIFIED BY 'hmon_password';
        Ignoring query to other database
        mysql> grant all on *.* TO 'hmon'@'ubuntucmbigdata1' IDENTIFIED BY 'hmon_password';
        Ignoring query to other database
        mysql> grant all on hmon.* TO 'hmon'@'%' IDENTIFIED BY 'hmon_password';
        Ignoring query to other database
        mysql> create database nav DEFAULT CHARACTER SET utf8;
        Ignoring query to other database
        mysql> grant all on nav.* TO 'nav'@'localhost' IDENTIFIED BY 'nav_password';
        Ignoring query to other database
        mysql> grant all on *.* TO 'nav'@'ubuntucmbigdata1' IDENTIFIED BY 'nav_password';
        Ignoring query to other database
        mysql> grant all on nav.* TO 'nav'@'%' IDENTIFIED BY 'nav_password';
        Ignoring query to other database
        mysql> flush privileges;
        mysql> exit;
        Bye
        root@ubuntucmbigdata1:~# 
        复制代码

         

    • 配置启动Cloudera Manager Server (只在ubuntucmbigdata1上执行)

      1. sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm scm_password

       

    • 也许大家会出现这个错误。

    • 复制代码
      root@ubuntucmbigdata1:~# sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm scm_password
      +======================================================================+
      |      Error: JAVA_HOME is not set and Java could not be found         |
      +----------------------------------------------------------------------+
      | Please download the latest Oracle JDK from the Oracle Java web site  |
      |  > http://www.oracle.com/technetwork/java/javase/index.html <        |
      |                                                                      |
      | Cloudera Manager requires Java 1.6 or later.                         |
      | NOTE: This script will find Oracle Java whether you install using    |
      |       the binary or the RPM based installer.                         |
      +======================================================================+
      root@ubuntucmbigdata1:~#
      复制代码

      则,需要安装jdk。

    • 成功的话,则是如下
    • 复制代码
      bigdata@ubuntucmbigdata1:/usr/java$ sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm scm_password
      JAVA_HOME=/usr/java/jdk1.7.0_80
      Verifying that we can write to /etc/cloudera-scm-server
      Creating SCM configuration file in /etc/cloudera-scm-server
      Executing:  /usr/java/jdk1.7.0_80/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
      [                          main] DbCommandExecutor              INFO  Successfully connected to database.
      All done, your SCM database is configured correctly!
      bigdata@ubuntucmbigdata1:/usr/java$ 
      复制代码

       

    • 安装JDK1.7 (所有节点)
      CDH要求至少是Oracle JDK7,Ubuntu自带的是OpenJDK,Ubuntu没有自带的Oracle JDK,必须添加外部源才能安装JDK。 
      ① 采用Cloudera软件源 
      在上一步配置Cloudera软件源后,可直接执行下面命令安装:

      1. sudo apt-get install oracle-j2sdk1.7

      ② 采用PPP源 
      添加PPA repository到系统,配置PPP源安装:

      1. sudo add-apt-repository ppa:webupd8team/java
      2. sudo apt-get update
      3. sudo apt-get upgrade
      4. sudo apt-get install oracle-java7-installer
      5. sudo apt-get install oracle-java7-set-default
      6. java -version

      注:若提示如下信息:

      1. sha256sum mismatch jdk-7u80-linux-x64.tar.gz
      2. Oracle JDK 7 is NOT installed.

      可能是jdk软件包没有下载完全,可执行如下命令删除后重新安装:

      1. sudo apt-get purge oracle-java7-installer

      ③ 手动安装 
      从Oracle官网下载JDK7u80:jdk-7u80-linux-x64.tar.gz,解压到/usr/java/jdk.1.7.0_80

      1. mkdir /usr/java
      2. sudo tar -zxvf jdk-7u80-linux-x64.tar.gz -/usr/java

      配置JDK环境变量: 

    • vim /etc/profile

      • source /etc/profile

        #jdk
        export JAVA_HOME=/usr/java/jdk1.7.0_80
        export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
        export PATH=$PATH:$JAVA_HOME/bin

         

 

  如果上述安装错了,查看和卸载命令。

Ubuntu系统里如何正确卸载oracle-j2sdk1.6(图文详解)

  则为

dpkg --list | grep -i jdk
sudo apt-get remove oracle-j2sdk1.6
sudo apt-get purge oracle-java7-set-default
sudo apt-get purge  oracle-java7-installer  

 

    安装orcal-java7-installer出错,则见

  orcal-java7-installer中断错误 CSDN

 

 

 

 

  •  sudo service cloudera-scm-server start(只在ubuntucmbigdata1上执行)
  • 成功,则出现
  • bigdata@ubuntucmbigdata1:/usr/java$ sudo service cloudera-scm-server start
    Starting cloudera-scm-server:  * cloudera-scm-server started
    
    bigdata@ubuntucmbigdata1:/usr/java$ 

     

  •  

    若启动报错:

    1. /etc/init.d/cloudera-scm-server: line 292: [[: [error] JVM PermGen is set less than 256m, CM server may run out of PermGen space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.: syntax error: operand expected (error token is "[error] JVM PermGen is set less than 256m, CM server may run out of PermGen space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.")
    2. /etc/init.d/cloudera-scm-server: line 292: [[: [error] JVM MaxHeap is set less than 2G, CM server may run out of Heap space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.: syntax error: operand expected (error token is "[error] JVM MaxHeap is set less than 2G, CM server may run out of Heap space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.")

    修改配置文件/etc/default/cloudera-scm-server,找到export CM_JAVA_OPTS部分,将-Xmx2G修改为-Xmx4G

  • 查看日志,验证是否成功启动

    1. tail -/var/log/cloudera-scm-server/cloudera-scm-server.log
 
 
 
 
 

安装Cloudera Manager Agents

    • 安装Cloudera Manager Agent软件包 
      在所有节点(包括ubuntucmbigdata1)上执行:

      1. sudo apt-get install cloudera-manager-agent cloudera-manager-daemons
    • 配置Cloudera Manager Agent 
      在所有节点上,修改配置文件sudo vi /etc/cloudera-scm-agent/config.ini,配置Cloudera Manager Server的主机和端口:

      1. server_host=ubuntucmbigdata1
      2. server_port=7182 #默认端口即可
    • 启动Cloudera Manager Agent(包含ubuntucmbigdata1)

      1. sudo service cloudera-scm-agent start

 

复制代码
bigdata@ubuntucmbigdata1:~$ sudo service cloudera-scm-agent start
cloudera-scm-agent is already running
bigdata@ubuntucmbigdata1:~$ 

bigdata@ubuntucmbigdata2:~$ sudo service cloudera-scm-agent start
cloudera-scm-agent is already running
bigdata@ubuntucmbigdata1:~$ 



bigdata@ubuntucmbigdata3:~$ sudo service cloudera-scm-agent start
cloudera-scm-agent is already running
bigdata@ubuntucmbigdata1:~$ 





bigdata@ubuntucmbigdata4:~$ sudo service cloudera-scm-agent start
cloudera-scm-agent is already running
bigdata@ubuntucmbigdata1:~$ 
复制代码

 

 

 

   

 

 

 

 

 

  

方式2、放在/opt/softwares/cm-5.6.0

参考

Cloudera Manager安装之Cloudera Manager 5.3.X安装(三)(tar方式、rpm方式和yum方式)

  

 

  先不用安装什么jdk,以及什么其他的。只如下执行就是

 

 

  执行cloudera-manager-installer.bin

复制代码
复制代码
复制代码
bigdata@ubuntucmbigdata1:~$ sudo mkdir -p /opt/softwares/cm-5.6.0
[sudo] password for bigdata: 
bigdata@ubuntucmbigdata1:~$ cd /opt/softwares/cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ pwd
/opt/softwares/cm-5.6.0
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 8
drwxr-xr-x 2 root root 4096  6月 25 22:19 ./
drwxr-xr-x 3 root root 4096  6月 25 22:19 ../
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chown -R bigdata:bigdata /opt/softwares/*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 8
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 22:19 ./
drwxr-xr-x 3 root    root    4096  6月 25 22:19 ../
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rw-r--r-- 1 root    root    517878  6月 25 22:17 cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chown -R bigdata:bigdata cloudera-manager-installer.bin 
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rw-r--r-- 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chmod +x ./cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rwxr-xr-x 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ 
复制代码
复制代码
复制代码

 




 

复制代码
复制代码
复制代码
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ pwd
/etc/apt/sources.list.d
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 12
drwxr-xr-x 2 root root 4096  6月 25 20:26 ./
drwxr-xr-x 6 root root 4096  6月 25 20:26 ../
-rw-r--r-- 1 root root  136  6月 25 20:26 webupd8team-java-trusty.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo curl "http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/cloudera.list" -o /etc/apt/sources.list.d/cloudera_precise.list
[sudo] password for bigdata: 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   263  100   263    0     0    422      0 --:--:-- --:--:-- --:--:--   422
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 16
drwxr-xr-x 2 root root 4096  6月 25 22:26 ./
drwxr-xr-x 6 root root 4096  6月 25 20:26 ../
-rw-r--r-- 1 root root  263  6月 25 22:26 cloudera_precise.list
-rw-r--r-- 1 root root  136  6月 25 20:26 webupd8team-java-trusty.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo curl -s http://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -
OK
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo apt-get update
复制代码
复制代码
复制代码

 

 

 

 

 

复制代码
bigdata@ubuntucmbigdata1:~$ cd /etc/apt/sources.list.d
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 16
drwxr-xr-x 2 root root 4096 Jun 26 11:10 ./
drwxr-xr-x 6 root root 4096 Jun 26 11:10 ../
-rw-r--r-- 1 root root  175 Oct 20  2015 cloudera-cm5.list
-rw-r--r-- 1 root root  263 Jun 26 11:10 cloudera_precise.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$
复制代码

 

 

 

 

 

bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rwxr-xr-x 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo ./cloudera-manager-installer.bin

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

   省略。

 

 

  也许中间,会报错,这个一一解决就是了

 

 

复制代码
bigdata@ubuntucmbigdata1:~$ cat /var/log/cloudera-manager-installer/3.install-oracle-j2sdk1.7.log
Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
  libntdb1 python-ntdb
Use 'apt-get autoremove' to remove them.
The following NEW packages will be installed:
  oracle-j2sdk1.7
0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded.
2 not fully installed or removed.
Need to get 142 MB of archives. After this operation, 292 MB of additional disk space will be used. Get:1 http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/ trusty-cm5/contrib oracle-j2sdk1.7 amd64 1.7.0+update67-1 [142 MB] Fetched 142 MB in 15min 9s (157 kB/s) Selecting previously unselected package oracle-j2sdk1.7. (Reading database ... 172281 files and directories currently installed.) Preparing to unpack .../oracle-j2sdk1.7_1.7.0+update67-1_amd64.deb ... Unpacking oracle-j2sdk1.7 (1.7.0+update67-1) ... Setting up oracle-java7-installer (7u80+7u60arm-0~webupd8~1) ... Downloading Oracle Java 7... --2017-06-26 00:20:41-- http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz Resolving download.oracle.com (download.oracle.com)... 23.2.16.218, 23.2.16.121 Connecting to download.oracle.com (download.oracle.com)|23.2.16.218|:80... connected. HTTP request sent, awaiting response... 302 Moved Temporarily Location: https://edelivery.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz [following] --2017-06-26 00:20:41-- https://edelivery.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz Resolving edelivery.oracle.com (edelivery.oracle.com)... 96.16.169.83, 2402:4f00:4001:197::2d3e, 2402:4f00:4001:180::2d3e Connecting to edelivery.oracle.com (edelivery.oracle.com)|96.16.169.83|:443... connected. HTTP request sent, awaiting response... 302 Moved Temporarily Location: http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz?AuthParam=1498407762_def59b5e0df725a8c8cee2b6d12ac74e [following] --2017-06-26 00:20:42-- http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz?AuthParam=1498407762_def59b5e0df725a8c8cee2b6d12ac74e Connecting to download.oracle.com (download.oracle.com)|23.2.16.218|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2017-06-26 00:20:43 ERROR 404: Not Found. download failed Oracle JDK 7 is NOT installed. dpkg: error processing package oracle-java7-installer (--configure): subprocess installed post-installation script returned error exit status 1 dpkg: dependency problems prevent configuration of oracle-java7-set-default: oracle-java7-set-default depends on oracle-java7-installer; however: Package oracle-java7-installer is not configured yet. dpkg: error processing package oracle-java7-set-default (--configure): dependency problems - leaving unconfigured Setting up oracle-j2sdk1.7 (1.7.0+update67-1) ... No apport report written because the error message indicates its a followup error from a previous failure. Errors were encountered while processing: oracle-java7-installer oracle-java7-set-default E: Sub-process /usr/bin/dpkg returned an 
复制代码

 

 

 

  解决这个问题,还是得自己手动下载来安装,才得以解决。

   它的下载链接

http://download.oracle.com/otn/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz

 

 

 

#java
export JAVA_HOME=/usr/java/jdk1.7.0_80
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin

 

 

 

 

 

bigdata@ubuntucmbigdata1:/usr/java$ sudo vim ~/.bashrc

 

 

 

 

 

 

总结

  对于这步安装jdk,需要注意,见

https://www.zybuluo.com/ncepuwanghui/note/474966

Ubuntu Server 14.04 安装部署 CDH5.7.2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

  这里,我也觉得很奇怪,在ubuntu系统里,是localhost,可我明明注销掉了。这跟centos下安装有点不同。

 

 

   但是呢,竟然可以用ubuntucmbigdata1:7810或者192.168.86.31来访问。

 

 

 

  当安装完成,Cloudera Manager 管理控制台会提供一个完成的url包括默认端口7180。(这个安装过程可能持续很长时间,网速较慢,可能会持续一上午,甚至一天)
这时候把这个地址输入浏览器地址栏,就能看到页面了。默认用户名和密码都为admin.

 

 

 

 

 

   成功!

 

 

   若是通过bin方式安装的话,则

 

复制代码
bigdata@ubuntucmbigdata1:/etc/init.d$ pwd
/etc/init.d
bigdata@ubuntucmbigdata1:/etc/init.d$ ls
acpid    apparmor      bluetooth            cloudera-scm-server-db  cups          dns-clean          halt        killprocs  mysql       ondemand    procps      rc.local  reboot      rsync    sendsigs  speech-dispatcher  thermald  umountnfs.sh         urandom
anacron  apport        brltty               console-setup           cups-browsed  friendly-recovery  irqbalance  kmod       networking  postgresql  pulseaudio  rcS       resolvconf  rsyslog  single    ssh                udev      umountroot           x11-common
apache2  avahi-daemon  cloudera-scm-server  cron                    dbus          grub-common        kerneloops  lightdm    ntp         pppd-dns    rc          README    rpcbind     saned    skeleton  sudo               umountfs  unattended-upgrades
bigdata@ubuntucmbigdata1:/etc/init.d$ sudo /etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:  * cloudera-scm-server started

bigdata@ubuntucmbigdata1:/etc/init.d$ sudo /etc/init.d/cloudera-scm-server-db start
 * Cloudera manager database already started
bigdata@ubuntucmbigdata1:/etc/init.d$ 
复制代码

 

 

 

 

 

复制代码
bigdata@ubuntucmbigdata3:~$ cd /etc/init.d/
bigdata@ubuntucmbigdata3:/etc/init.d$ ls
acpid    apparmor      bluetooth           console-setup  cups-browsed  friendly-recovery  irqbalance  kmod        ntp       procps      rc.local  reboot      rsync    sendsigs  speech-dispatcher  thermald  umountnfs.sh         urandom
anacron  apport        brltty              cron           dbus          grub-common        kerneloops  lightdm     ondemand  pulseaudio  rcS       resolvconf  rsyslog  single    ssh                udev      umountroot           x11-common
apache2  avahi-daemon  cloudera-scm-agent  cups           dns-clean     halt               killprocs   networking  pppd-dns  rc          README    rpcbind     saned    skeleton  sudo               umountfs  unattended-upgrades
bigdata@ubuntucmbigdata3:/etc/init.d$ sudo ./cloudera-scm-agent start
cloudera-scm-agent is already running
bigdata@ubuntucmbigdata3:/etc/init.d$ 
复制代码

 

 

 

 

 

 

 

 

 

 

 

 方式3,通过tar,gz方法

其实很简单,网上写的资料各种杂活。就是如下!(博主,我是很喜欢这个人写的这篇博客)!!!!

在线和离线安装Cloudera CDH 5.6.0

 

  的确,我也跟这位博主一样,也需要在实验室里部署。以及也是4台机器。(至于原因,大家都懂得,中国高校研究生喜欢4、3、2、1来进行加速比做实验论文)

 

     在实验室安装过社区版的Hadoop,当时用的是普通PC机,安装的过程遇到了好多问题。当解决了大部分的安装问题之后由于实验室的需求,需要安装一个商业发行版的hadoop ,我们就采用的Cloudera 公司发行的版本。首先要说明的是CDH对内存要求比较高,当我们采用原来的4G内存的PC机,只要启动基本内存就满了,并且会拖慢整个集群的运算速度。所以说安装CDH对硬件的要求比较高。安装CDH 有三种安装方式:在线安装、半离线安装、离线安装。在线安装和离线安装我都安装过,虽然在线安装貌似简单但是实际用起来还是没有离线安装方便,我还是建议大家采用离线安装的方式。不过下面都会介绍在线安装的方式和离线安装的方式。希望对别人有所帮助。

 

   关于,前期的介绍、硬件基础、安装Ubuntu14.04系统、设置hosts、关闭防火墙、设置静态IP、安装Oracle JDK 8(所有节点)、所有节点免密码SSH登陆、建立数据库(仅主机)、设置集群时间同步(所有节点)。我都上面已经做好了。

 

 

 

相关软件和下载

Cloudera Manager : http://archive.cloudera.com/cm5/cm/5/   
Cloudera CDH:http://archive.cloudera.com/cdh5 
MySql的JDBC驱动jar包 

 

 

  因为,我这里是ubuntu14.04,所以

 

 

 

  同理,我这里,也不去一枚地追求新,采用的是5.6.0

 

 

 

   这个网上多的是,大家随便去下载就是了

 

 

 

 

 

 

 


根据系统下载所相应的版本,下载的文件如下

cloudera-manager-trusty-cm5.6.0_amd64.tar.gz
CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel
CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel.sha
manifest.json 
mysql-connector-java-5.1.32-bin.jar

 

 

 

 

 

   

  解压 cloudera-manager-trusty-cm5.6.0_amd64.tar.gz 到 /opt 目录下

   先弄清楚,为什么这么做,是因为,在线安装的话,默认是安装在/opt下。所以,我这里带大家离线安装Cloudera安装搭建部署大数据集群,更是如此,要放在/opt下了。

 

 

 

 

 

复制代码
bigdata@ubuntucmbigdata1:/opt$ pwd
/opt
bigdata@ubuntucmbigdata1:/opt$ ll
total 8
drwxr-xr-x  2 root root 4096  2月 18  2016 ./
drwxr-xr-x 23 root root 4096  6月 24 15:09 ../
bigdata@ubuntucmbigdata1:/opt$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt$ ll
total 505696
drwxr-xr-x  2 root root      4096  6月 25 18:37 ./
drwxr-xr-x 23 root root      4096  6月 24 15:09 ../
-rw-r--r--  1 root root 517818680  6月 25 18:22 cloudera-manager-trusty-cm5.6.0_amd64.tar.gz
bigdata@ubuntucmbigdata1:/opt$ sudo tar -zxvf cloudera-manager-trusty-cm5.6.0_amd64.tar.gz 
复制代码

 

 

 

复制代码
复制代码
bigdata@ubuntucmbigdata1:/opt$ pwd
/opt
bigdata@ubuntucmbigdata1:/opt$ ll
total 505700
drwxr-xr-x  3 1106  592      4096  2月 12  2016 ./
drwxr-xr-x 23 root root      4096  6月 24 15:09 ../
-rw-r--r--  1 root root 517818680  6月 25 18:22 cloudera-manager-trusty-cm5.6.0_amd64.tar.gz
drwxr-xr-x  8 1106  592      4096  2月 12  2016 cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt$ sudo rm cloudera-manager-trusty-cm5.6.0_amd64.tar.gz 
bigdata@ubuntucmbigdata1:/opt$ sudo chown -R bigdata:bigdata cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt$ ll
total 12
drwxr-xr-x  3    1106     592 4096  6月 25 18:46 ./
drwxr-xr-x 23 root    root    4096  6月 24 15:09 ../
drwxr-xr-x  8 bigdata bigdata 4096  2月 12  2016 cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt$ 
复制代码
复制代码

 

 

 

 

 

  添加mysql connector包到/opt/cm-5.6.0/share/cmf/lib下

 

 

复制代码
复制代码
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/share/cmf/lib$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/share/cmf/lib$ ls
activation-1.1.1.jar                       commons-fileupload-1.2.2.jar       gson-2.2.jar                                   jaxb-impl-2.1.13.jar                     mx4j-3.0.1.jar                            spring-security-crypto-3.1.0.RELEASE.jar
agent-5.6.0.jar                            commons-httpclient-3.1.jar         guava-14.0.jar                                 jaxp-api-1.4.jar                         mx4j-tools-3.0.1.jar                      spring-security-ldap-3.1.0.RELEASE.jar
alarms-5.6.0.jar                           commons-io-2.4.jar                 hawtjni-runtime-1.8.jar                        jboss-logging-3.1.0.GA.jar               mysql-connector-java-5.1.32-bin.jar       spring-security-saml2-core-1.0.1.RELEASE.jar
alertpublisher-5.6.0.jar                   commons-lang-2.6.jar               headlamp-5.6.0.jar                             jcommander-1.30.jar                      nav-common-5.6.0.jar                      spring-security-web-3.1.0.RELEASE.jar
复制代码
复制代码

 

 

 

 

   下面,参考这篇博客,非常方便

Cloudera Manager 5和CDH5离线安装

 

 

 

创建用户(所有节点)

  创建cloudera-scm用户 
   即,在我这里就是,

 

bigdata@ubuntucmbigdata1:~$ sudo useradd --system --home=/opt/cm-5.3.3/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

   我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

 

 

 

 

 

 

   初始化CM5的数据库数据库(j仅ubuntucmbigdata1节点)

复制代码
复制代码
bigdata@ubuntucmbigdata1:~$ sudo /opt/cm-5.6.0/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -prootroot --scm-host localhost scm scm scm
+======================================================================+
|      Error: JAVA_HOME is not set and Java could not be found         |
+----------------------------------------------------------------------+
| Please download the latest Oracle JDK from the Oracle Java web site  |
|  > http://www.oracle.com/technetwork/java/javase/index.html <        |
|                                                                      |
| Cloudera Manager requires Java 1.6 or later.                         |
| NOTE: This script will find Oracle Java whether you install using    |
|       the binary or the RPM based installer.                         |
+======================================================================+
bigdata@ubuntucmbigdata1:~$ 
复制代码
复制代码

 

 

 

 

 

Agent 配置

   修改/opt/cm-5.6.0/etc/cloudera-scm-agent/config.ini中的server_host为主节点的主机名。 

bigdata@ubuntucmbigdata1:~$ sudo vim /opt/cm-5.6.0/etc/cloudera-scm-agent/config.ini

 

 

 

 

 

 

复制代码
[General]
# Hostname of the CM server.
server_host=ubuntucmbigdata1

# Port that the CM server is listening on.
server_port=7182
复制代码

 

 

 

 

 

 

Parcels离线包设置,用以安装(仅主节点)

  将下载好的

   CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel 
 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel.sha 
manifest.json

      这三个文件移至/opt/cloudera/parcel-repo/中,没有这个目录的话,手动创建。

 

 

 

 

 

复制代码
复制代码
bigdata@ubuntucmbigdata1:/opt$ pwd
/opt
bigdata@ubuntucmbigdata1:/opt$ ll
total 12
drwxr-xr-x  3    1106     592 4096  6月 25 18:46 ./
drwxr-xr-x 23 root    root    4096  6月 24 15:09 ../
drwxr-xr-x  8 bigdata bigdata 4096  2月 12  2016 cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt$ sudo mkdir -p /opt/cloudera/parcel-repo/
bigdata@ubuntucmbigdata1:/opt$ cd /opt/cloudera/parcel-repo/
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ pwd
/opt/cloudera/parcel-repo
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 8
drwxr-xr-x 2 root root 4096  6月 25 19:43 ./
drwxr-xr-x 3 root root 4096  6月 25 19:43 ../
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ 
复制代码
复制代码

 

 

 

 

 

 

复制代码
复制代码
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ pwd
/opt/cloudera/parcel-repo
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 8
drwxr-xr-x 2 root root 4096  6月 25 19:43 ./
drwxr-xr-x 3 root root 4096  6月 25 19:43 ../
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo chown -R bigdata:bigdata /opt/cloudera/parcel-repo/
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 8
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:43 ./
drwxr-xr-x 3 root    root    4096  6月 25 19:43 ../
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 1497448
drwxr-xr-x 2 bigdata bigdata       4096  6月 25 19:48 ./
drwxr-xr-x 3 root    root          4096  6月 25 19:43 ../
-rw-r--r-- 1 root    root    1533315179  6月 25 18:31 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel
-rw-r--r-- 1 root    root            41  6月 25 18:22 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel.sha1
-rw-r--r-- 1 root    root         49913  6月 25 18:32 manifest.json
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo chown -R bigdata:bigdata /opt/cloudera/parcel-repo/*
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 1497448
drwxr-xr-x 2 bigdata bigdata       4096  6月 25 19:48 ./
drwxr-xr-x 3 root    root          4096  6月 25 19:43 ../
-rw-r--r-- 1 bigdata bigdata 1533315179  6月 25 18:31 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel
-rw-r--r-- 1 bigdata bigdata         41  6月 25 18:22 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel.sha1
-rw-r--r-- 1 bigdata bigdata      49913  6月 25 18:32 manifest.json
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ 
复制代码
复制代码

 

 

 

 

 

相关启动脚本 
通过/opt/cm-5.3.3/etc/init.d/cloudera-scm-server start启动服务端。 
通过/opt/cm-5.3.3/etc/init.d/cloudera-scm-agent start启动Agent服务。 
我们启动的其实是个service脚本,需要停止服务将以上的start参数改为stop就可以了,重启是restart。 

 

复制代码
复制代码
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo /opt/cm-5.6.0/etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:  * Couldn't start cloudera-scm-server

bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo /opt/cm-5.6.0/etc/init.d/cloudera-scm-agent start 
Starting cloudera-scm-agent:  * cloudera-scm-agent started

bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo /opt/cm-5.6.0/etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:  * Couldn't start cloudera-scm-server

bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ 
复制代码
复制代码

 

   查看日志

注:查看安装目录下log文件夹下cloudera-scm-agent/cloudera-scm-agent.out 看Agent 是否启动成功 。 

 

复制代码
复制代码
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ pwd
/opt/cm-5.6.0/log
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ l
cloudera-scm-agent/  cloudera-scm-server/  cmf/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ ll
total 20
drwxr-xr-x 5 bigdata bigdata 4096  2月 12  2016 ./
drwxr-xr-x 8 bigdata bigdata 4096  2月 12  2016 ../
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:53 cloudera-scm-agent/
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:52 cloudera-scm-server/
drwxr-xr-x 2 bigdata bigdata 4096  2月 12  2016 cmf/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ cd cloudera-scm-server/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-server$ ll
total 12
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:52 ./
drwxr-xr-x 5 bigdata bigdata 4096  2月 12  2016 ../
-rw-r--r-- 1 root    root     730  6月 25 19:53 cloudera-scm-server.out
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-server$ cat cloudera-scm-server.out 
+======================================================================+
|      Error: JAVA_HOME is not set and Java could not be found         |
+----------------------------------------------------------------------+
| Please download the latest Oracle JDK from the Oracle Java web site  |
|  > http://www.oracle.com/technetwork/java/javase/index.html <        |
|                                                                      |
| Cloudera Manager requires Java 1.6 or later.                         |
| NOTE: This script will find Oracle Java whether you install using    |
|       the binary or the RPM based installer.                         |
+======================================================================+
复制代码
复制代码

   查看日志,可以看出,JAVA_HOME,没有设置。(是这种对ubuntu系统不适用,但是对centos系统是可以的。无解)

 

 

 

 

 

然后,发现,得用

   原来,在

在线和离线安装Cloudera CDH 5.6.0

  里

 

 

 
bigdata@ubuntucmbigdata1:~$ sudo add-apt-repository ppa:webupd8team/java

复制代码
复制代码
bigdata@ubuntucmbigdata1:~$ sudo add-apt-repository ppa:webupd8team/java
 Oracle Java (JDK) Installer (automatically downloads and installs Oracle JDK7 / JDK8 / JDK9). There are no actual Java files in this PPA.

Important -> Why Oracle Java 7 And 6 Installers No Longer Work: http://www.webupd8.org/2017/06/why-oracle-java-7-and-6-installers-no.html

More info (and Ubuntu installation instructions):
- for Oracle Java 7: http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html
- for Oracle Java 8: http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html

Debian installation instructions:
- Oracle Java 7: http://www.webupd8.org/2012/06/how-to-install-oracle-java-7-in-debian.html
- Oracle Java 8: http://www.webupd8.org/2014/03/how-to-install-oracle-java-8-in-debian.html

Oracle Java 9 (for both Ubuntu and Debian): http://www.webupd8.org/2015/02/install-oracle-java-9-in-ubuntu-linux.html

For JDK9, the PPA uses standard builds from: http://jdk.java.net/9/ (and not the Jigsaw builds!).

Important!!! For now, you should continue to use Java 8 because Oracle Java 9 is available as an early access release! You should only use Oracle Java 9 if you explicitly need it, because it may contain bugs and it might not include the latest security patches! Also, some Java options were removed in JDK9, so you may encounter issues with various Java apps. More information and installation instructions (Ubuntu / Linux Mint / Debian): http://www.webupd8.org/2015/02/install-oracle-java-9-in-ubuntu-linux.html
 More info: https://launchpad.net/~webupd8team/+archive/ubuntu/java
Press [ENTER] to continue or ctrl-c to cancel adding it (回车)

gpg: keyring `/tmp/tmpel2bk0b8/secring.gpg' created
gpg: keyring `/tmp/tmpel2bk0b8/pubring.gpg' created
gpg: requesting key EEA14886 from hkp server keyserver.ubuntu.com
gpg: /tmp/tmpel2bk0b8/trustdb.gpg: trustdb created
gpg: key EEA14886: public key "Launchpad VLC" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
OK
bigdata@ubuntucmbigdata1:~$ 
复制代码
复制代码

    我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

 

 

 

 




bigdata@ubuntucmbigdata1:~$ sudo apt-get update

    我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

 

 

 

 

 

 

 

 

 

 

复制代码
bigdata@ubuntucmbigdata1:~$ sudo apt-get upgrade
...
Do you want to continue? [Y/n] y
...
*** sudoers (Y/I/N/O/D/Z) [default=N] ? y

复制代码

 

 

 

 

 

 

 

bigdata@ubuntucmbigdata1:~$ sudo apt-get install oracle-java7-installer
[sudo] password for bigdata: 
Do you want to continue? [Y/n] y

 

 

 

 

 

 

 

 

 

 

bigdata@ubuntucmbigdata1:~$ sudo apt-get install oracle-java7-set-default

 

 

 

 

 

  最后,我们需要如下,就可以了

复制代码
bigdata@ubuntucmbigdata1:/etc/cloudera-scm-server$ dpkg --list | grep -i jdk
iF  oracle-java7-installer                                7u80+7u60arm-0~webupd8~1                            all          Oracle Java(TM) Development Kit (JDK) 7
iU  oracle-java7-set-default                              7u80+7u60arm-0~webupd8~1                            all          Set Oracle JDK 7 as default Java
bigdata@ubuntucmbigdata1:/etc/cloudera-scm-server$ 
复制代码

 

   如果有多余的,如1.6版本的话,则

Ubuntu系统里如何正确卸载oracle-j2sdk1.6(图文详解)

 

 

 

 

 

 

 

 

 

 

 

bigdata@ubuntucmbigdata1:~$ sudo /opt/cm-5.6.0/etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:  * cloudera-scm-server started

bigdata@ubuntucmbigdata1:~$ 

   成功!

 

 

 

 

 

 

 

   查看cloudera-scm-agent的运行日志

复制代码
复制代码
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ ll
total 20
drwxr-xr-x 5 bigdata bigdata 4096  2月 12  2016 ./
drwxr-xr-x 8 bigdata bigdata 4096  2月 12  2016 ../
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:53 cloudera-scm-agent/
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:52 cloudera-scm-server/
drwxr-xr-x 2 bigdata bigdata 4096  2月 12  2016 cmf/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ cd cloudera-scm-agent/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-agent$ ll
total 40
drwxr-xr-x 2 bigdata bigdata  4096  6月 25 19:53 ./
drwxr-xr-x 5 bigdata bigdata  4096  2月 12  2016 ../
-rw-r--r-- 1 root    root    14757  6月 25 19:57 cloudera-scm-agent.log
-rw-r--r-- 1 root    root      935  6月 25 19:53 cloudera-scm-agent.out
-rw-r--r-- 1 root    root      202  6月 25 19:53 cmf_listener.log
-rw-r--r-- 1 root    root      539  6月 25 19:53 supervisord.log
-rw-r--r-- 1 root    root      449  6月 25 19:53 supervisord.out
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-agent$ cat cloudera-scm-agent.out 
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     SCM Agent Version: 5.6.0
[25/Jun/2017 19:53:01 +0000] 9651 MainThread util         INFO     Generating Agent UUID for the first time: '/opt/cm-5.6.0/lib/cloudera-scm-agent/uuid'
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     Adding env vars that start with CMF_AGENT_
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     explicitly passing env var to supervisee: MGMT_HOME: /opt/cm-5.6.0/share/cmf
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     explicitly passing env var to supervisee: ROOT: /opt/cm-5.6.0/lib/cmf
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     Logging to /opt/cm-5.6.0/log/cloudera-scm-agent/cloudera-scm-agent.log
Error: could not find config file /opt/cm-5.6.0/run/cloudera-scm-agent/supervisor/supervisord.conf
For help, use /opt/cm-5.6.0/lib/cmf/agent/src/cmf/agent.py -h
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-agent$ 
复制代码
复制代码

 

 

 

 


当cloudera-scm-server、cloudera-scm-agent start启动成功之后,可通过http://主节点IP:7180来访问CM,用户名密码都为admin。剩下的就是一直点下一步和根据具体的需求来配置集群了。

 

 

 

 

   然后,到这一步,我的

cloudera manager的7180 web界面访问不了的解决办法(图文详解)

 

  转换成,在线安装

  如下

  放在

/opt/softwares/cm-5.6.0

复制代码
复制代码
bigdata@ubuntucmbigdata1:~$ sudo mkdir -p /opt/softwares/cm-5.6.0
[sudo] password for bigdata: 
bigdata@ubuntucmbigdata1:~$ cd /opt/softwares/cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ pwd
/opt/softwares/cm-5.6.0
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 8
drwxr-xr-x 2 root root 4096  6月 25 22:19 ./
drwxr-xr-x 3 root root 4096  6月 25 22:19 ../
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chown -R bigdata:bigdata /opt/softwares/*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 8
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 22:19 ./
drwxr-xr-x 3 root    root    4096  6月 25 22:19 ../
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rw-r--r-- 1 root    root    517878  6月 25 22:17 cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chown -R bigdata:bigdata cloudera-manager-installer.bin 
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rw-r--r-- 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chmod +x ./cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rwxr-xr-x 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ 
复制代码
复制代码

 




 

复制代码
复制代码
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ pwd
/etc/apt/sources.list.d
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 12
drwxr-xr-x 2 root root 4096  6月 25 20:26 ./
drwxr-xr-x 6 root root 4096  6月 25 20:26 ../
-rw-r--r-- 1 root root  136  6月 25 20:26 webupd8team-java-trusty.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo curl "http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/cloudera.list" -o /etc/apt/sources.list.d/cloudera_precise.list
[sudo] password for bigdata: 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   263  100   263    0     0    422      0 --:--:-- --:--:-- --:--:--   422
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 16
drwxr-xr-x 2 root root 4096  6月 25 22:26 ./
drwxr-xr-x 6 root root 4096  6月 25 20:26 ../
-rw-r--r-- 1 root root  263  6月 25 22:26 cloudera_precise.list
-rw-r--r-- 1 root root  136  6月 25 20:26 webupd8team-java-trusty.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo curl -s http://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -
OK
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo apt-get update
复制代码

 

 

  本文转自大数据躺过的坑博客园博客,原文链接:http://www.cnblogs.com/zlslch/p/7078317.html,如需转载请自行联系原作者

相关实践学习
如何在云端创建MySQL数据库
开始实验后,系统会自动创建一台自建MySQL的 源数据库 ECS 实例和一台 目标数据库 RDS。
全面了解阿里云能为你做什么
阿里云在全球各地部署高效节能的绿色数据中心,利用清洁计算为万物互联的新世界提供源源不断的能源动力,目前开服的区域包括中国(华北、华东、华南、香港)、新加坡、美国(美东、美西)、欧洲、中东、澳大利亚、日本。目前阿里云的产品涵盖弹性计算、数据库、存储与CDN、分析与搜索、云通信、网络、管理与监控、应用服务、互联网中间件、移动服务、视频服务等。通过本课程,来了解阿里云能够为你的业务带来哪些帮助 &nbsp; &nbsp; 相关的阿里云产品:云服务器ECS 云服务器 ECS(Elastic Compute Service)是一种弹性可伸缩的计算服务,助您降低 IT 成本,提升运维效率,使您更专注于核心业务创新。产品详情: https://www.aliyun.com/product/ecs
相关文章
|
2月前
|
并行计算 Ubuntu Linux
Ubuntu学习笔记(五):18.04安装多版本CUDA
这篇博客文章介绍了在Ubuntu 18.04系统上如何安装和切换不同版本的CUDA,以及如何安装不同版本的cuDNN。
228 2
|
2月前
|
并行计算 PyTorch TensorFlow
Ubuntu安装笔记(一):安装显卡驱动、cuda/cudnn、Anaconda、Pytorch、Tensorflow、Opencv、Visdom、FFMPEG、卸载一些不必要的预装软件
这篇文章是关于如何在Ubuntu操作系统上安装显卡驱动、CUDA、CUDNN、Anaconda、PyTorch、TensorFlow、OpenCV、FFMPEG以及卸载不必要的预装软件的详细指南。
4118 3
|
19天前
|
Ubuntu 开发工具 git
Ubuntu安装homebrew的完整教程
本文介绍了如何在没有公网的情况下安装 Homebrew。首先访问 Homebrew 官网,然后通过阿里云的镜像克隆安装脚本,并创建普通用户进行安装。接着修改 `install.sh` 文件指向国内镜像,执行安装命令。最后配置环境变量并更换 Homebrew 源为国内镜像,确保安装顺利。
143 50
|
2月前
|
Ubuntu Linux 测试技术
Linux系统之Ubuntu安装cockpit管理工具
【10月更文挑战第13天】Linux系统之Ubuntu安装cockpit管理工具
151 4
Linux系统之Ubuntu安装cockpit管理工具
|
2月前
|
Ubuntu 应用服务中间件 nginx
Ubuntu安装笔记(三):ffmpeg(3.2.16)源码编译opencv(3.4.0)
本文是关于Ubuntu系统中使用ffmpeg 3.2.16源码编译OpenCV 3.4.0的安装笔记,包括安装ffmpeg、编译OpenCV、卸载OpenCV以及常见报错处理。
165 2
Ubuntu安装笔记(三):ffmpeg(3.2.16)源码编译opencv(3.4.0)
|
2月前
|
Ubuntu Linux C语言
Ubuntu安装笔记(二):ubuntu18.04编译安装opencv 3.4.0 opencv_contrib3.4.0
本文介绍了在Ubuntu 18.04系统上编译安装OpenCV 3.4.0及其扩展包opencv_contrib 3.4.0的详细步骤,包括下载源码、安装依赖、配置CMake和编译安装,以及常见问题的解决方法。
134 1
Ubuntu安装笔记(二):ubuntu18.04编译安装opencv 3.4.0 opencv_contrib3.4.0
|
2月前
|
Ubuntu 虚拟化
软件安装(二):VMware ubuntu20.04 安装步骤
这篇文章是关于如何在VMware Workstation 16 Player上安装Ubuntu 20.04桌面版的详细步骤指南。
191 2
软件安装(二):VMware ubuntu20.04 安装步骤
|
2月前
|
PyTorch TensorFlow 算法框架/工具
Jetson环境安装(一):Ubuntu18.04安装pytorch、opencv、onnx、tensorflow、setuptools、pycuda....
本文提供了在Ubuntu 18.04操作系统的NVIDIA Jetson平台上安装深度学习和计算机视觉相关库的详细步骤,包括PyTorch、OpenCV、ONNX、TensorFlow等。
56 1
Jetson环境安装(一):Ubuntu18.04安装pytorch、opencv、onnx、tensorflow、setuptools、pycuda....
|
2月前
|
消息中间件 监控 Ubuntu
大数据-54 Kafka 安装配置 环境变量配置 启动服务 Ubuntu配置 ZooKeeper
大数据-54 Kafka 安装配置 环境变量配置 启动服务 Ubuntu配置 ZooKeeper
83 3
大数据-54 Kafka 安装配置 环境变量配置 启动服务 Ubuntu配置 ZooKeeper
|
2月前
|
Kubernetes Ubuntu Docker
从0开始搞K8S:使用Ubuntu进行安装(环境安装)
通过上述步骤,你已经在Ubuntu上成功搭建了一个基本的Kubernetes单节点集群。这只是开始,Kubernetes的世界广阔且深邃,接下来你可以尝试部署应用、了解Kubernetes的高级概念如Services、Deployments、Ingress等,以及探索如何利用Helm等工具进行应用管理,逐步提升你的Kubernetes技能树。记住,实践是最好的老师,不断实验与学习,你将逐渐掌握这一强大的容器编排技术。
179 1