开发者社区> 问答> 正文

sparkstreaming 写数据到 phoenix报错问题

hbase小能手 2018-11-08 10:44:31 817

tf.foreachRDD( rdd => {
rdd.saveAsTextFile("hdfs://hacluster/user/comms/test")
val str1 = "NeCpuState"
val str2 = "DeviceID"
rdd.filter(!_.contains(str1)).filter(!_.contains(str2)).map(_.replaceAll(""", "")).map(_.split(",")).filter(_.length == 7) .foreach(line => {
val row = line(0).substring(3).reverse + MD5Util.stringMD5(UUID.randomUUID().toString)
val DeviceID = line(1)
val DeviceName = line(2)
val CollectionTime = line(3)
val GranularityPeriod = line(4)
val cpuUsage = line(5)
val sql = "upsert into TEST (ROW,DEVICEID,DEVICENAME,COLLECTIONTIME,GRANULARITYPERIOD,CPUUSAGE) values (" + row + "," + DeviceID + "," + DeviceName + "," + CollectionTime + "," + GranularityPeriod + "," + cpuUsage + ")"

execute_sql(sql) }) })
ssc }

def execute_sql(sql: String): Unit ={
val conn = DriverManager.getConnection("jdbc:phoenix:190.15.119.5,190.15.119.6,190.15.119.7")
val preStat = conn.prepareStatement(sql)
preStat.executeUpdate()
conn.commit()
preStat.close()
conn.close() }

报错

Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException

    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
    at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:448)
    at org.apache.hadoop.hbase.client.ConnectionManager.createConnectionInternal(ConnectionManager.java:357)
    at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:144)
    at org.apache.phoenix.query.HConnectionFactory$HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:286)
    ... 25 more

Caused by: java.lang.reflect.InvocationTargetException

    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
    ... 30 more

Caused by: java.lang.UnsupportedOperationException: Unable to find org.apache.hadoop.hbase.ipc.controller.ClientRpcControllerFactory

    at org.apache.hadoop.hbase.util.ReflectionUtils.instantiateWithCustomCtor(ReflectionUtils.java:36)
    at org.apache.hadoop.hbase.ipc.RpcControllerFactory.instantiate(RpcControllerFactory.java:58)
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:685)
    ... 35 more

Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.ipc.controller.ClientRpcControllerFactory

提交语句已经使用 --jars /opt/hadoopclient/HBase/hbase/lib/phoenix-core-4.4.0-HBase-1.0.jar
不知道为什么还是找不到这个类。

SQL 分布式计算 Java 数据库连接 分布式数据库 Hbase
分享到
取消 提交回答
全部回答(1)
  • 马铭芳
    2019-07-17 23:12:53

    hbase-protcol.java这个包存在冲突。

    1 0
+ 订阅

分享数据库前沿,解构实战干货,推动数据库技术变革

推荐文章
相似问题