一、安装：（注意版本兼容）

hadoop2.7.2，hive1.3，spark1.6

相关配置：

参考上面
**把hive-site.xml、core-site.xml、hdfs-site.xml放入main下面的resource中
file->Project settings->Modules->把resource变成source**

三、编程：

import org.apache.log4j.{Level, Logger}
import org.apache.spark.sql.SparkSession
import org.apache.spark.{SparkConf, SparkContext}

object SparkOnHive {
  def main(args: Array[String]): Unit = {
    Logger.getLogger("org").setLevel(Level.ERROR)
    val conf: SparkConf = new SparkConf().setAppName(this.getClass.getSimpleName).setMaster("local[2]")
    val sc: SparkContext = new SparkContext(conf)

    val spark: SparkSession = SparkSession.builder() .appName(this.getClass.getSimpleName)
      .master("local[2]")
      .enableHiveSupport()  //支持hive
      .getOrCreate()

    var sql = "show databases"
    spark.sql(sql).show()

    sql = "select * from log.stu_info"
    spark.sql(sql).show()
    spark.stop()
  }
}

四、执行

1. 在hive上执行：

nohup hive --service metastore > metastore.log 2>&1 &

1. 执行脚本

/home/hadoop/apps/spark/bin/spark-submit \
--class spark.day14.Demo02SparkOnHive \
--master spark://hadoop01:7077 \
--executor-memory 512m \
--total-executor-cores 2 \
--files /home/hadoop/apps/hive/conf/hive-site.xml \
--driver-class-path /home/hadoop/apps/hive/lib/mysql-connector-java-5.0.8-bin.jar \
/home/hadoop/sparkdata/spark2.1-2.0.jar

1. 在spark上执行：

会报警告：但不影响结果