在部署好CDH5(HDFS+MapReduce+Zookeeper)后,某一天想要通过web界面来添加Hive或者Oozie服务。
添加Hive组件服务:
1.报错:Unexpected error.Unable to verify database connection
2.解决办法(曲线救国的办法):
a.先选hive的服务节点,从namenode节点改为cm节点,这样就会在cm节点上创建角色
b.等部署完毕
c.进hive tab页,迁移角色或者删除角色,重新添加
添加Oozie组件服务:
1.又报错:Unexpected error.Unable to verify database connection
2.解决办法(根源方法)
[root@hadoopdn-04 ~]# tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
2016-02-09 13:20:44,062 INFO 67058165@scm-web-132:com.cloudera.server.cmf.cluster.RulesCluster: Done with service and role creation
2016-02-09 13:21:25,474 INFO 1079246313@scm-web-136:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing host command OozieTestDatabaseConnection BasicCmdArgs{args=[hadoopnn-01, mysql, hadoopdn-04, oozie, oozie, oozie_password]}. Host: DbHost{id=1, hostId=4f4999cd-42fd-4f9f-949e-c9581ed98d7f, hostName=hadoopnn-01}
2016-02-09 13:21:25,667 WARN 264839353@agentServer-98:com.cloudera.server.cmf.AgentProtocolImpl: Received Process Heartbeat for unknown (or duplicate) process. Ignoring. This is expected to happen once after old process eviction or process deletion (as happens in restarts).id=775 name=null host=4f4999cd-42fd-4f9f-949e-c9581ed98d7f/hadoopnn-01
2016-02-09 13:21:27,380 INFO CommandPusher:com.cloudera.cmf.service.AbstractOneOffHostCommand: Unsuccessful 'OozieTestDatabaseConnection'
2016-02-09 13:21:27,381 INFO CommandPusher:com.cloudera.cmf.service.AbstractDbConnectionTestCommand: Command exited with code: 1
2016-02-09 13:21:27,381 INFO CommandPusher:com.cloudera.cmf.service.AbstractDbConnectionTestCommand: + local MISCJAVA_HOME_CANDIDATES
+ case ${BIGTOP_JAVA_MAJOR} in
+ JAVA_HOME_CANDIDATES=(${JAVA7_HOME_CANDIDATES[@]} ${JAVA8_HOME_CANDIDATES[@]} ${JAVA6_HOME_CANDIDATES[@]} ${MISCJAVA_HOME_CANDIDATES[@]} ${OPENJAVA7_HOME_CANDIDATES[@]} ${OPENJAVA8_HOME_CANDIDATES[@]} ${OPENJAVA6_HOME_CANDIDATES[@]})
+ '[' -z '' ']'
+ for candidate_regex in '${JAVA_HOME_CANDIDATES[@]}'
++ ls -rvd /usr/java/jdk1.7.0_40 /usr/java/jdk1.7.0_67-cloudera
+ for candidate in '`ls -rvd ${candidate_regex}* 2>/dev/null`'
+ '[' -e /usr/java/jdk1.7.0_67-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
+ break 2
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_67-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
+ source_parcel_environment
+ '[' '!' -z '' ']'
+ JAVA=/usr/java/jdk1.7.0_67-cloudera/bin/java
+ [[ -z /usr/share/cmf ]]
+ MGMT_CLASSPATH='/usr/share/cmf/lib/*'
+ JDBC_JARS=/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-jdbc.jar:/usr/share/java/oracle-connector-java.jar
+ MGMT_CLASSPATH='/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-jdbc.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*'
++ pwd
+ MGMT_CLASSPATH='/var/run/cloudera-scm-agent/process/795-OOZIE.OOZIE_SERVER-test-db-connection:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-jdbc.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*'
+ MGMT_JAVA_OPTS='-Djava.net.preferIPv4Stack=true '
+ exec /usr/java/jdk1.7.0_67-cloudera/bin/java -Djava.net.preferIPv4Stack=true -Djava.security.egd=file:///dev/urandom -cp '/var/run/cloudera-scm-agent/process/795-OOZIE.OOZIE_SERVER-test-db-connection:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-jdbc.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*' com.cloudera.enterprise.dbutil.DbCommandExecutor db.properties
Error: Could not find or load main class com.cloudera.enterprise.dbutil.DbCommandExecutor
###################################################################################################################################
当前情况:在hadoopnn-01(namenode)部署Oozie服务,而数据库在hadoopdn-04(cm)上(使用mysql数据库).
分析以下这句报错及图01(报错信息太过模糊,无法准确定位错误,所以还是要善于使用 tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log):
+ exec /usr/java/jdk1.7.0_67-cloudera/bin/java -Djava.net.preferIPv4Stack=true -Djava.security.egd=file:///dev/urandom -cp '/var/run/cloudera-scm-agent/process/795-OOZIE.OOZIE_SERVER-test-db-connection:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-jdbc.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*' com.cloudera.enterprise.dbutil.DbCommandExecutor db.properties
经过分析验证:
1. /usr/share/java/mysql-connector-java.jar 路径存在(网上好多报该错误时,给的是这个mysql-connector-java.jar没有在相应路径配置,纯属瞎扯淡。假如没有这个jar,报错就不是这个错误了,会非常明显的)
2. /usr/share/cmf/lib/* ,该路径不存在,因为这个 cmf/lib/* 是部署在cm机器上的,只要对应的从cm机器scp到hadoopnn-01机器的相同路径即可
然后,在去验证与mysql是否连接上,如图02(
Successful)。
备注: 之前我单独添加hive服务时,也是这个错误,只不过当时采取了一种曲线救国的方式。
添加Hive组件服务:
1.报错:Unexpected error.Unable to verify database connection
2.解决办法(曲线救国的办法):
a.先选hive的服务节点,从namenode节点改为cm节点,这样就会在cm节点上创建角色
b.等部署完毕
c.进hive tab页,迁移角色或者删除角色,重新添加
添加Oozie组件服务:
1.又报错:Unexpected error.Unable to verify database connection
2.解决办法(根源方法)
[root@hadoopdn-04 ~]# tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
2016-02-09 13:20:44,062 INFO 67058165@scm-web-132:com.cloudera.server.cmf.cluster.RulesCluster: Done with service and role creation
2016-02-09 13:21:25,474 INFO 1079246313@scm-web-136:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing host command OozieTestDatabaseConnection BasicCmdArgs{args=[hadoopnn-01, mysql, hadoopdn-04, oozie, oozie, oozie_password]}. Host: DbHost{id=1, hostId=4f4999cd-42fd-4f9f-949e-c9581ed98d7f, hostName=hadoopnn-01}
2016-02-09 13:21:25,667 WARN 264839353@agentServer-98:com.cloudera.server.cmf.AgentProtocolImpl: Received Process Heartbeat for unknown (or duplicate) process. Ignoring. This is expected to happen once after old process eviction or process deletion (as happens in restarts).id=775 name=null host=4f4999cd-42fd-4f9f-949e-c9581ed98d7f/hadoopnn-01
2016-02-09 13:21:27,380 INFO CommandPusher:com.cloudera.cmf.service.AbstractOneOffHostCommand: Unsuccessful 'OozieTestDatabaseConnection'
2016-02-09 13:21:27,381 INFO CommandPusher:com.cloudera.cmf.service.AbstractDbConnectionTestCommand: Command exited with code: 1
2016-02-09 13:21:27,381 INFO CommandPusher:com.cloudera.cmf.service.AbstractDbConnectionTestCommand: + local MISCJAVA_HOME_CANDIDATES
+ case ${BIGTOP_JAVA_MAJOR} in
+ JAVA_HOME_CANDIDATES=(${JAVA7_HOME_CANDIDATES[@]} ${JAVA8_HOME_CANDIDATES[@]} ${JAVA6_HOME_CANDIDATES[@]} ${MISCJAVA_HOME_CANDIDATES[@]} ${OPENJAVA7_HOME_CANDIDATES[@]} ${OPENJAVA8_HOME_CANDIDATES[@]} ${OPENJAVA6_HOME_CANDIDATES[@]})
+ '[' -z '' ']'
+ for candidate_regex in '${JAVA_HOME_CANDIDATES[@]}'
++ ls -rvd /usr/java/jdk1.7.0_40 /usr/java/jdk1.7.0_67-cloudera
+ for candidate in '`ls -rvd ${candidate_regex}* 2>/dev/null`'
+ '[' -e /usr/java/jdk1.7.0_67-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
+ break 2
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_67-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
+ source_parcel_environment
+ '[' '!' -z '' ']'
+ JAVA=/usr/java/jdk1.7.0_67-cloudera/bin/java
+ [[ -z /usr/share/cmf ]]
+ MGMT_CLASSPATH='/usr/share/cmf/lib/*'
+ JDBC_JARS=/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-jdbc.jar:/usr/share/java/oracle-connector-java.jar
+ MGMT_CLASSPATH='/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-jdbc.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*'
++ pwd
+ MGMT_CLASSPATH='/var/run/cloudera-scm-agent/process/795-OOZIE.OOZIE_SERVER-test-db-connection:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-jdbc.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*'
+ MGMT_JAVA_OPTS='-Djava.net.preferIPv4Stack=true '
+ exec /usr/java/jdk1.7.0_67-cloudera/bin/java -Djava.net.preferIPv4Stack=true -Djava.security.egd=file:///dev/urandom -cp '/var/run/cloudera-scm-agent/process/795-OOZIE.OOZIE_SERVER-test-db-connection:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-jdbc.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*' com.cloudera.enterprise.dbutil.DbCommandExecutor db.properties
Error: Could not find or load main class com.cloudera.enterprise.dbutil.DbCommandExecutor
###################################################################################################################################
当前情况:在hadoopnn-01(namenode)部署Oozie服务,而数据库在hadoopdn-04(cm)上(使用mysql数据库).
分析以下这句报错及图01(报错信息太过模糊,无法准确定位错误,所以还是要善于使用 tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log):
+ exec /usr/java/jdk1.7.0_67-cloudera/bin/java -Djava.net.preferIPv4Stack=true -Djava.security.egd=file:///dev/urandom -cp '/var/run/cloudera-scm-agent/process/795-OOZIE.OOZIE_SERVER-test-db-connection:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-jdbc.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*' com.cloudera.enterprise.dbutil.DbCommandExecutor db.properties
经过分析验证:
1. /usr/share/java/mysql-connector-java.jar 路径存在(网上好多报该错误时,给的是这个mysql-connector-java.jar没有在相应路径配置,纯属瞎扯淡。假如没有这个jar,报错就不是这个错误了,会非常明显的)
2. /usr/share/cmf/lib/* ,该路径不存在,因为这个 cmf/lib/* 是部署在cm机器上的,只要对应的从cm机器scp到hadoopnn-01机器的相同路径即可
点击(此处)折叠或打开
- [root@hadoopnn-01 ~]# cd /usr/share/
- [root@hadoopnn-01 share]# mkdir -p cmf/lib
- [root@hadoopdn-04 ~]# scp /usr/share/cmf/lib/* root@hadoopnn-01:/usr/share/cmf/lib
备注: 之前我单独添加hive服务时,也是这个错误,只不过当时采取了一种曲线救国的方式。