浅析 hive udaf 的正确编写方式- 论姿势的重要性

select my_udaf_sample(colA,colB,colC,'/user/userA/conf/') as my_udaf_result from tableA a group by a.cust_id;

org.apache.hadoop.hive.ql.metadata.HiveException: 读取文件[configure-fileA.xml]异常：java.io.FileNotFoundException: File does not exist: /user/hive/configure-fileA.xml；

1. private static String udfFilePath = ""; //定义静态变量并赋初始值
2. public void iterate(AggregationBuffer aggregationBuffer, Object[] objects) ：// iterate 方法中对静态变量重新赋值
  if (StringUtils.isBlank(udfFilePath)) {
     udfFilePath = (String) stringOI.getPrimitiveJavaObject(objects[13]);
   }   
3. public Object terminate(AggregationBuffer aggregationBuffer)：// terminate 方法中使用该静态变量
  InputStream ips = HdfsUtil.getConfFileStream(FILE_NAME, udfFilePath);
4. HdfsUtil.getConfFileStream(): //HdfsUtil读取配置文件内容
  FileSystem fs = FileSystem.get(URI.create(udfFilePath + fileName),conf);
 FSDataInputStream inputStream = fs.open(new Path(udfFilePath + fileName));

A Generic User-defined aggregation function (GenericUDAF) for the use with Hive. New GenericUDAF classes need to inherit from this GenericUDAF class. The GenericUDAF are superior to normal UDAFs in the following ways: 1. It can accept arguments of complex types, and return complex types. 2. It can accept variable length of arguments. 3. It can accept an infinite number of function signature - for example, it's easy to write a GenericUDAF that accepts array , array  > and so on (arbitrary levels of nesting).

1. public static enum Mode {  
2.     /** 
3.      * PARTIAL1: This is the map phase of mapreduce: from raw data to partial data aggregation 
4.      * iterate() and terminatePartial() will be called 
5.      */  
6.     PARTIAL1,  
7.         /** 
8.      * PARTIAL2: This is the map-side Combiner phase of mapreduce, responsible for merging map data at the map-side: from partial data aggregation to partial data aggregation: 
9.      * merge() and terminatePartial() will be called 
10.      */  
11.     PARTIAL2,  
12.         /** 
13.      * FINAL: mapreduce Reduction phase: from partial data aggregation to complete aggregation 
14.      * merge() and terminate() will be called 
15.      */  
16.     FINAL,  
17.         /** 
18.      * COMPLETE: If this stage occurs, it means that map reduce has only map, not reduce, so the map side will produce the result directly: from the original data directly to the complete aggregation. 
19.       * iterate() and terminate() will be called 
20.      */  
21.     COMPLETE  
22.   };