开发者社区> 问答> 正文

使用flink的kafka到opentsdb:java.lang.NoClassDefFoundError:无法初始化类org.apache.http.conn.ssl.SSLConnectionSocketFactory

flink小助手 2018-11-28 16:28:41 1527

"这是我的代码:

def saveToOpenTSDB(rows: Iterator[String], url: String) {

      val requestConfig: RequestConfig = RequestConfig.custom().setSocketTimeout(2000).setConnectTimeout(2000).setConnectionRequestTimeout(2000).build()
      val httpClient: CloseableHttpClient = HttpClients.createDefault() }

错误堆栈跟踪:

2018-10-25 12:40:39,323 INFO org.apache.flink.client.cli.CliFrontend - --------------------------- -------------------------------------------------- --- 2018-10-25 12:40:39,324 INFO org.apache.flink.client.cli.CliFrontend - 启动命令行客户端(版本:1.6.1,Rev:23e2636,日期:14.09.2018 @ 19:56 :46 UTC)2018-10-25 12:40:39,324 INFO org.apache.flink.client.cli.CliFrontend - 操作系统当前用户:root 2018-10-25 12:40:39,676 INFO org.apache.flink.client .cli.CliFrontend - 当前Hadoop / Kerberos用户:root 2018-10-25 12:40:39,676 INFO org.apache.flink.client.cli.CliFrontend - JVM:Java HotSpot(TM)64位服务器VM - Oracle Corporation - 1.8 / 25.172-b11 2018-10-25 12:40:39,676 INFO org.apache.flink.client.cli。CliFrontend - 最大堆大小:7136个MiBytes 2018年10月25日12:40:39676 INFO org.apache.flink.client.cli.CliFrontend - JAVA_HOME:/usr/java/jdk1.8.0_172 2018年10月25日12:40 :39,678 INFO org.apache.flink.client.cli.CliFrontend - Hadoop版本:2.6.5 2018-10-25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend - JVM选项:2018-10 -25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend - 5 2018-10-25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend - JVM选项:2018-10-25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend - 5 2018-10-25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend - JVM选项:2018-10-25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend -
-Dlog.file = / root / flink-1.6.1 / log / flink-root-client-cuiyk-cdn-test-10.log 2018-10-25 12:40:39,678 INFO org.apache.flink.client。 cli.CliFrontend - -Dlog4j.configuration =文件:2018年10月25日/root/flink-1.6.1/conf/log4j-cli.properties 12:40:39678 INFO org.apache.flink.client.cli.CliFrontend - -Dlogback.configurationFile = file:/root/flink-1.6.1/conf/logback.xml 2018-10-25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend - Program Arguments:2018-10 -25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend -
运行2018-10-25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend - -c 2018-10-25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend - dataclean .FlinkDataCleanDemo 2018-10-25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend -
--parallelism 2018-10-25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend - 4 2018-10-25 12:40:39,678 INFO org.apache.flink.client.cli.CliFrontend - --jobmanager 2018-10-25 12:40:39,679 INFO org.apache.flink.client.cli.CliFrontend -
cuiyk-cdn-test-4:42115 2018-10-25 12:40:39,679 INFO org.apache.flink.client.cli.CliFrontend -
./flinkkafka2tsdb.jar 2018-10-25 12:40:39,679 INFO org.apache.flink.client.cli.CliFrontend - Classpath:/root/flink-1.6.1/lib/flink-python_2.11-1.6.1的.jar:/root/flink-1.6.1/lib/flink-shaded-hadoop2-uber-1.6.1.jar:/root/flink-1.6.1/lib/log4j-1.2.17.jar:/根/弗林克-1.6.1 / lib目录/ SLF4J-log4j12-1.7.7.jar:/root/flink-1.6.1/lib/flink-dist_2.11-1.6.1.jar :: /选择/ Cloudera公司/包裹/ CDH -5.13.3-1.cdh5.13.3.p0.2 / lib / hadoop / etc / hadoop:2018-10-25 12:40:39,679 INFO org.apache.flink.client.cli.CliFrontend - ---- -------------------------------------------------- -------------------------- 2018-10-25 12:40:39,682 INFO org.apache.flink.configuration.GlobalConfiguration - 加载配置属性:classloader.resolve-order,parent-first 2018-10-25 12:40:39,682 INFO org.apache.flink。configuration.GlobalConfiguration - 加载配置属性:jobmanager.rpc.address,localhost 2018-10-25 12:40:39,682 INFO org.apache.flink.configuration.GlobalConfiguration - 加载配置属性:jobmanager.rpc.port,6123 2018-10 -25 12:40:39,682 INFO org.apache.flink.configuration.GlobalConfiguration - 加载配置属性:jobmanager.heap.size,1024m 2018-10-25 12:40:39,682 INFO org.apache.flink.configuration.GlobalConfiguration - 加载配置属性:taskmanager.heap.size,1024m 2018-10-25 12:40:39,683 INFO org.apache.flink.configuration.GlobalConfiguration - 加载配置属性:taskmanager.numberOfTaskSlots,1 2018-10-25 12:40: 39,683 INFO org.apache.flink.configuration。GlobalConfiguration - 加载配置属性:parallelism.default,1 2018-10-25 12:40:39,683 INFO org.apache.flink.configuration.GlobalConfiguration - 加载配置属性:rest.port,8081 2018-10-25 12:40: 39,696 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli

  • 在/tmp/.yarn-properties-root下找到Yarn属性文件。2018-10-25 12:40:39,914 INFO org.apache.flink.runtime.security.modules.HadoopModule - Hadoop用户设置为root(auth:SIMPLE)2018-10-25 12:40:39,936 INFO org.apache。 flink.client.cli.CliFrontend - 运行'run'命令。2018-10-25 12:40:39,940 INFO org.apache.flink.client.cli.CliFrontend - 从JAR文件构建程序2018-10-25 12:40:40,314 INFO org.apache.flink.runtime.rest.RestClient - Rest客户端端点已启动。2018-10-25 12:40:40,316 INFO org.apache.flink.client.cli.CliFrontend - 开始执行程序2018-10-25 12:40:40,316 INFO org.apache.flink.client.program.rest。GlobalConfiguration - 加载配置属性:rest.port,8081 2018-10-25 12:40:40,540 INFO org.apache.flink.client.program.rest.RestClusterClient - 提交作业1b45b5780e0e7067fae20be1db8e7c2f(分离:false)。2018-10-25 12:41:38,177 INFO org.apache.flink.runtime.rest.RestClient - 关闭休止端点。2018-10-25 12:41:38,179 INFO org.apache.flink.runtime.rest.RestClient - Rest端点关闭完成。2018-10-25 12:41:38,180 ERROR org.apache.flink.client.cli.CliFrontend - 运行命令时出错。org.apache.flink.client.program.ProgramInvocationException:作业失败。(JobID:1b45b5780e0e7067fae20be1db8e7c2f)org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:

//related dependecies:

<dependency>
        <groupId>commons-httpclient</groupId>
        <artifactId>commons-httpclient</artifactId>
        <version>3.0</version>
    </dependency>

    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient</artifactId>
        <version>4.4</version>
    </dependency>"
消息中间件 分布式计算 资源调度 Oracle Hadoop Java 关系型数据库 Kafka 网络架构 流计算 实时计算(流计算)
分享到
取消 提交回答
全部回答(1)
  • flink小助手
    2019-07-17 23:16:51

    "我刚刚解决了这个问题,在'配置'标签中添加了maven shade插件,如下所示:

                    <relocations>
                        <relocation>
                            <pattern>org.apache.http</pattern>
                            <shadedPattern>shade.org.apache.http</shadedPattern>
                        </relocation>
                    </relocations>"
    0 0
阿里云实时计算
使用钉钉扫一扫加入圈子
+ 订阅

一套基于Apache Flink构建的一站式、高性能实时大数据处理平台,广泛适用于流式数据处理、离线数据处理、DataLake计算等场景。

推荐文章
相似问题
链接