文章目录
前言
Apache Hive是一个分布式、容错的数据仓库系统，能够支持大规模的分析。Hive元数据仓库(HMS)提供了一个中央的元数据存储库，可轻松分析数据以做出明智的数据驱动决策，因此它是许多数据湖架构的关键组件。Hive建立在Apache Hadoop之上，支持在S3、adls、gs等存储上通过HDFS访问。Hive允许用户使用SQL读取、写入和管理PB级别的数据。
1.安装mysql
1）使用yum安装
yum install mysql-server -y
2) 安装好启动服务
service mysqld start
3) 开启开机自动启动
chkconfig mysqld on
4) 修改权限
grant all privileges on *.* to 'root'@'%' identified by '123456' with grant option
5) 登录
mysql -uroot -p
2.Hive安装 Local/Embedded Metastore Database (Derby)
Installing Hive from a Stable Release 可以参考官网文档详细的安装步骤
1) 下载稳定版本解压即可
此处安装的是hive-2.3.9
# 解压
tar -xf apache-hive-2.3.9-bin.tar.gz
# 修改名字
mv apache-hive-2.3.9-bin hive-2.3.9
2）配置环境变量
Set the environment variable HIVE_HOME to point to the installation directory:
vi /etc/profile
#添加
export HIVE_HOME=/opt/bigdata/hive-2.3.9
export PATH=$HIVE_HOME/bin:$PATH
3) 配置远程连接mysql数据库
需要在hive安装路径下conf
cd /opt/bigdata/hive-2.3.9/conf
mv hive-default.xml.template hive-site.xml
vi hive-site.xml
# 需要配置configuration标签之外的配置
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://node01:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>123456</value>
</property>
4）需要将mysql驱动添加到lib下
mysql-connector-java-5.1.32-bin.jar
/opt/bigdata/hive-2.3.9/lib
5) 初始化元数据
# `schematool -dbType mysql -initSchema`
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/bigdata/hive-2.3.9/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/bigdata/hadoop-2.6.5/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://node01:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: root
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.mysql.sql
Initialization script completed
schemaTool completed
6)登录mysql进行查看
3.远程服务安装 Remote Metastore Database
选择一个两个节点，一个hive作为客户端[【node01】，一个hive作为服务端【node02】连接mysql ,
环境变量HIVE_HOME配置不在赘述。
1 ）服务端的hive-site.xml配置
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive_remote/warehouse</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://node01:3306/hive_remote?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>123</value>
</property>
2) 客户端需要配置连接服务端
Metastore server and client communicate using Thrift Protocol 。
客户端配置
hive.metastore.local = false Metastore is remote. Note: This is no longer needed as of Hive 0.10.
hive.metastore.uris = thrift://<host_name>:<port>host and port for the Thrift metastore server.
hive.metastore.warehouse.dir=<base hdfs path>
客户端hive-site.xml的配置
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive_remote/warehouse</value>
</property>
<property>
<name>hive.metastore.uris</name>
<value>thrift://node02:9083</value>
</property>
3)在服务端初始化元数据
schematool -dbType mysql -initSchema
4) 服务端启动服务
hive --service metastore
5）客户端连接
hive
