前言
看我往期文章的朋友都会发现我聊SQL的时候喜欢拿一份Hive的源码来做参考,其实在我看来,对于技术人员来说,源码其实就是最好的参考资料了,不管是网络上面有的或者没有的,源码都可以给你最原汁原味的解释。Hive版本3.x其实出来很久了,本文结合编译的过程,顺便聊聊编译的那些事。
准备工作
在入手一份源码之前,首先要知道源码是从哪里可以搞到,hive代码首先是可以在github上面下载的,搜索关键字 "hive github "
其实发现cdh的版本也是可以搜索到,问题不大!
当然hive源码包也有自己的地址的:
http://archive.apache.org/dist/hive/hive-3.0.0/
把包拿过来即可。
编译
编译这个过程事先学习一下hive的源码结构,我们在pom.xml下面找到这么一些字样,这个其实代表了我们可以不同的编译选项,里面可以在我们参数中加上:
<profiles> <profile> <id>thriftif</id> ...... </build> </profile> <profile> <id>sources</id> ...... </profile> <profile> <id>javadoc</id> ...... </profile> <profile> <id>findbugs</id> ...... </build> <reporting> ...... </profile> <profile> <!-- Windows-specific settings to allow unit tests to work --> <id>windows-test</id> <activation> <os> <family>Windows</family> </os> </activation> ...... </build> </profiles>
我组合出来的命令如下:
mvn clean package -Pdist -DskipTests -Dmaven.javadoc.skip=true 这个命令就是对源码解释就是 打包-Pdist, 跳过单元测试-DskipTests 跳过文档生成 -Dmaven.javadoc.skip=true 这样子一套比较高效的组合就可以了
接下来就是编译过程
可能出现的问题有以下原因:
1、默认的maven仓库不能用,表现为网络不通,这个建议切换到阿里云的网络环境下面就好
2、包冲突、这个可能是本机有一些包没同步,这个建议先有一份干净的环境
剩下的就是结果了,我这边目测都比较顺利 ^^
[INFO] Reactor Summary for Hive 3.1.2: [INFO] [INFO] Hive Upgrade Acid .................................. SUCCESS [ 5.376 s] [INFO] Hive ............................................... SUCCESS [ 0.276 s] [INFO] Hive Classifications ............................... SUCCESS [ 0.463 s] [INFO] Hive Shims Common .................................. SUCCESS [ 1.712 s] [INFO] Hive Shims 0.23 .................................... SUCCESS [ 2.198 s] [INFO] Hive Shims Scheduler ............................... SUCCESS [ 1.254 s] [INFO] Hive Shims ......................................... SUCCESS [ 0.931 s] [INFO] Hive Common ........................................ SUCCESS [ 6.785 s] [INFO] Hive Service RPC ................................... SUCCESS [ 2.745 s] [INFO] Hive Serde ......................................... SUCCESS [ 4.828 s] [INFO] Hive Standalone Metastore .......................... SUCCESS [ 27.272 s] [INFO] Hive Metastore ..................................... SUCCESS [ 3.241 s] [INFO] Hive Vector-Code-Gen Utilities ..................... SUCCESS [ 0.398 s] [INFO] Hive Llap Common ................................... SUCCESS [ 3.268 s] [INFO] Hive Llap Client ................................... SUCCESS [ 2.517 s] [INFO] Hive Llap Tez ...................................... SUCCESS [ 2.838 s] [INFO] Hive Spark Remote Client ........................... SUCCESS [ 3.582 s] [INFO] Hive Query Language ................................ SUCCESS [ 47.177 s] [INFO] Hive Llap Server ................................... SUCCESS [ 5.972 s] [INFO] Hive Service ....................................... SUCCESS [ 6.418 s] [INFO] Hive Accumulo Handler .............................. SUCCESS [ 4.462 s] [INFO] Hive JDBC .......................................... SUCCESS [ 17.497 s] [INFO] Hive Beeline ....................................... SUCCESS [ 3.844 s] [INFO] Hive CLI ........................................... SUCCESS [ 3.014 s] [INFO] Hive Contrib ....................................... SUCCESS [ 2.307 s] [INFO] Hive Druid Handler ................................. SUCCESS [02:31 min] [INFO] Hive HBase Handler ................................. SUCCESS [ 53.683 s] [INFO] Hive JDBC Handler .................................. SUCCESS [ 6.051 s] [INFO] Hive HCatalog ...................................... SUCCESS [ 26.763 s] [INFO] Hive HCatalog Core ................................. SUCCESS [ 18.106 s] [INFO] Hive HCatalog Pig Adapter .......................... SUCCESS [ 3.760 s] [INFO] Hive HCatalog Server Extensions .................... SUCCESS [ 31.195 s] [INFO] Hive HCatalog Webhcat Java Client .................. SUCCESS [ 3.653 s] [INFO] Hive HCatalog Webhcat .............................. SUCCESS [ 15.729 s] [INFO] Hive HCatalog Streaming ............................ SUCCESS [ 4.264 s] [INFO] Hive HPL/SQL ....................................... SUCCESS [ 11.598 s] [INFO] Hive Streaming ..................................... SUCCESS [ 2.985 s] [INFO] Hive Llap External Client .......................... SUCCESS [ 3.088 s] [INFO] Hive Shims Aggregator .............................. SUCCESS [ 0.152 s] [INFO] Hive Kryo Registrator .............................. SUCCESS [ 2.426 s] [INFO] Hive TestUtils ..................................... SUCCESS [ 0.234 s] [INFO] Hive Packaging ..................................... SUCCESS [ 55.682 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 09:11 min [INFO] Finished at: 2021-11-07T16:43:29+08:00```
导入Idea
一份热气腾腾的源码环境就搞好了~~