环境
阿里云ECS服务器
hadoop-2.7.2
apache-hive-1.2.1
注意、注意、注意：我的hive软件装在 /opt/module/ 下，注意一下下面代码执行的目录
前提
请记住快照一下，无论文章写的在差，也有人点赞，无论文章写的在好，也还是有人看不懂，如果您没有搭建起来，也不至于毁你的环境
请记住快照一下，无论文章写的在差，也有人点赞，无论文章写的在好，也还是有人看不懂，如果您没有搭建起来，也不至于毁你的环境
请记住快照一下，无论文章写的在差，也有人点赞，无论文章写的在好，也还是有人看不懂，如果您没有搭建起来，也不至于毁你的环境
hadoop安装成功并且运行起来，yarn必须启动起来
apache-hive-1.2.1-bin.tar.gz下载
链接：https://pan.baidu.com/s/1l6t2a3xL2f09GXfKOVJyng
提取码：adsw
复制这段内容后打开百度网盘手机App，操作更方便哦
安装步骤
将Hive的压缩包上传到Linux系统中的/opt/software中
将Hive的压缩包解压到/opt/module中
tar -zxvf apache-hive-1.2.1-bin.tar.gz -C /opt/module/
进入 /opt/module中，并且将解压的的文件名称改为hive(方便后面写名字)
[root@iZm5ea99qngm2v98asii1aZ module]# cd /opt/module/ [root@iZm5ea99qngm2v98asii1aZ module]# ls apache-hive-1.2.1-bin hadoop-2.7.2 [root@iZm5ea99qngm2v98asii1aZ module]# mv apache-hive-1.2.1-bin/ hive [root@iZm5ea99qngm2v98asii1aZ module]# ls hadoop-2.7.2 hive [root@iZm5ea99qngm2v98asii1aZ module]#
修改 /exc/profile 文件，配置Hive的环境变量
1. #HIVE_HOME 2. export HIVE_HOME=/opt/module/hive 3. export PATH=$PATH:$HIVE_HOME/bin
修改前
修改后
使配置文件生效
source /etc/profile
在hive中配置hadoop的HADOOP_HOME和HIVE_CONF_DIR
进入/opt/module/hive/conf 目录
将 hive-env.sh.template 复制一份 并且命名为 hive-env.sh
修改hive-env.sh文件
其中 HADOOP_HOME的获取方式为
HIVE_CONF_DIR的值为修改的这个文件的目录
修改前
修改后
在HDFS上创建几个Hive规定的文件
hadoop fs -mkdir /tmp hadoop fs -mkdir -p /user/hive/warehouse hadoop fs -chmod g+w /tmp hadoop fs -chmod g+w /user/hive/warehouse
启动Hive
进入/opt/module/hive/ 目录下
cd /opt/module/hive/
启动hive
bin/hive
退出hive(后面带分号)
quit;
Hive安装MySQL服务:将hive的数据存储到MySQL中，将hive和MySQL结合起来
安装MySQL
安装方式一：
https://blog.csdn.net/qq_37171353/article/details/80040662
安装方式二：
用宝塔界面 宝塔面板 - 简单好用的Linux/Windows服务器运维管理面板
步骤一：安装宝塔界面,注意看里面最后生成的用户名、密码、URL
步骤二：和手机应用宝一样安装MySQL
配置MySQL无主机登录
目的
解决在别的电脑上也能能连接到服务器MySQL
解决办法
grant all privileges on *.* to root@'%' identified by 'root';
flush privileges;
这样你就可以在任何机器上输入正确的ip,用户名，密码就可以访问数据库了，前提其他都没有问题（mysql服务开启，端口开放等等）
修改hive的配置文件
在hive-site.xml添加下面信息
其中需要修改三处信息：jdbcURL,MySQL的用户名、密码
<value>jdbc:mysql://127.0.0.1:3306/metastore?createDatabaseIfNotExist=true</value> <value>root</value> <value>000000</value>
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://127.0.0.1:3306/metastore?createDatabaseIfNotExist=true</value> <description>JDBC connect string for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> <description>Driver class name for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> <description>username to use against metastore database</description> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>000000</value> <description>password to use against metastore database</description> </property> <property> <name>hive.cli.print.header</name> <value>true</value> </property> <property> <name>hive.cli.print.current.db</name> <value>true</value> </property> </configuration>
上传MySQL驱动（mysql-connector-java-5.1.27-bin.jar）
将mysql-connector-java-5.1.27-bin上传到 （ /opt/module/hive/lib） 下
mysql-connector-java-5.1.27-bin.jar下载链接
链接：https://pan.baidu.com/s/1-qfK3nwrm07tg6zVKk1Vrw
提取码：737r
复制这段内容后打开百度网盘手机App，操作更方便哦
初始化schematool（敲黑板）
schematool -dbType mysql -initSchema
否则会报下面的错误
hive配置远程仓库异常：Unable to instantiate
Logging initialized using configuration in jar:file:/bigdata/hive/apache-hive-1.2.1-bin/lib/hive-common-1.2.1.jar!/hive-log4j.properties Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:239) at org.apache.hadoop.util.RunJar.main(RunJar.java:153) Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503) ... 8 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521) ... 14 more Caused by: javax.jdo.JDOException: Exception thrown calling table.exists() for hive.`SEQUENCE_TABLE` NestedThrowables: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:596) at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732) at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752) at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:6773) at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6670) at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6645) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114) at com.sun.proxy.$Proxy6.verifySchema(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:572) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:624) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
大功告成
启动hive
hive运行日志信息配置
在 （/opt/module/hive/conf） 里有 hive-log4j.properties.template 复制一份并且重命名为hive-log4j.properties
修改hive-log4j.properties文件
修改前
修改后
