AI 助理
备案控制台

游客guzk5azdrhbmu_个人页

游客guzk5azdrhbmu
个人头像照片
文章
0
问答
3
视频
0

个人介绍

暂无个人介绍

擅长的技术

获得更多能力
通用技术能力：

暂时未有相关通用技术能力~

云产品技术能力：

阿里云技能认证

详细说明
暂无更多信息

2021年09月

2020年01月

正在加载, 请稍后...
暂无更多信息
  • 回答了问题 2021-09-03

    如何计算spark Scala中2行之间的时间差

    val sparkSession = SparkSession .builder() .appName(this.getClass.getSimpleName) //本地测试 .master('local[4]') .config('spark.driver.host', '127.0.0.1') .config('spark.driver.bindAddress', '127.0.0.1') .getOrCreate() val rows = Seq( Row('31-AUG-21 02.16.33.371 PM') ) val schema = StructType( Seq( StructField('my_time', StringType, true) ) ) val rowsRDD = sparkSession.sparkContext.parallelize(rows, 4) val df = sparkSession.createDataFrame(rowsRDD, schema) df.withColumn('time1',expr('date_format(from_unixtime(unix_timestamp(my_time,'dd-MMM-yy hh.mm.ss.SSS a'),'yyyy-MM-dd HH:mm:ss.SSS'),'yyyy-MM-dd HH:mm:ss.SSS')')) .show(10)
    踩0 评论0
  • 回答了问题 2020-01-11

    flink on yarn错误“Yarn只有-1个虚拟核心可用”

    请问flink1.9.0-cdh6.3.0从：https://www.cloudera.com/downloads/cdf/csa-trial.html处下载安装运行在cdh6.3.0中后，只能运行：flink run -m yarn-cluster -yn 4 -yjm 1024 -ytm 1024 /opt/cloudera/parcels/FLINK-1.9.0-csa1.0.0.0-cdh6.3.0/lib/flink/examples/streaming/WordCount.jar --input hdfs://master:8020/user/root/testflink.txt --output hdfs://master:8020/user/root/flink/wordcount_output，该安装包下所提供的其他示例，如：flink run /opt/cloudera/parcels/FLINK-1.9.0-csa1.0.0.0-cdh6.3.0/lib/flink/examples/streaming/SocketWindowWordCount.jar --port 9009却报错见：https://developer.aliyun.com/ask/133744?spm=a2c6h.13159736，而由于我cdh未启用kerberos安全认证功能，故我在cdh的flink“配置”处清除了默认的两个选项共三个默认值：Client和KafkaClient、flink.keytab，重启后运行flink list命令仍报错：WARN zookeeper.ClientCnxn: SASL configuration failed。请问如何弃用此安全功能？
    踩0 评论0
  • 回答了问题 2020-01-09

    Flink配置Kerberos认证问题

    org.apache.flink.util.FlinkException: Failed to retrieve job list. at org.apache.flink.client.cli.CliFrontend.listJobs(CliFrontend.java:445) at org.apache.flink.client.cli.CliFrontend.lambda$list$0(CliFrontend.java:427) at org.apache.flink.client.cli.CliFrontend.runClusterAction(CliFrontend.java:956) at org.apache.flink.client.cli.CliFrontend.list(CliFrontend.java:424) at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1024) at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1096) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875) at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1096) Caused by: java.util.concurrent.TimeoutException at org.apache.flink.runtime.concurrent.FutureUtils$Timeout.run(FutureUtils.java:998) at org.apache.flink.runtime.concurrent.DirectExecutorService.execute(DirectExecutorService.java:211) at org.apache.flink.runtime.concurrent.FutureUtils.lambda$orTimeout$14(FutureUtils.java:416) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
    踩0 评论1
正在加载, 请稍后...
滑动查看更多
正在加载, 请稍后...
暂无更多信息