开发者社区> 问答> 正文

在idea中跑spark自带的例子SparkPi报错,请大神帮忙看一下,谢谢。?报错

在idea中跑spark自带的例子SparkPi报错,请大神帮忙看一下,谢谢。

日志如下,不知道是哪里配置错了

15/11/23 22:29:39 INFO executor.CoarseGrainedExecutorBackend: Registered signal handlers for [TERM, HUP, INT]
15/11/23 22:29:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/11/23 22:29:41 INFO spark.SecurityManager: Changing view acls to: spark,zcy
15/11/23 22:29:41 INFO spark.SecurityManager: Changing modify acls to: spark,zcy
15/11/23 22:29:41 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spark, zcy); users with modify permissions: Set(spark, zcy)
15/11/23 22:29:42 INFO slf4j.Slf4jLogger: Slf4jLogger started
15/11/23 22:29:42 INFO Remoting: Starting remoting
15/11/23 22:29:42 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://driverPropsFetcher@SparkWorker203:42203]
15/11/23 22:29:42 INFO util.Utils: Successfully started service 'driverPropsFetcher' on port 42203.
15/11/23 22:29:43 WARN Remoting: Tried to associate with unreachable remote address [akka.tcp://sparkDriver@DESKTOP-5HOMAO8:7265]. Address is now gated for 5000 ms, all messages to this address will be delivered to dead letters. Reason: 拒绝连接: DESKTOP-5HOMAO8/192.168.40.111:7265
Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1643)
	at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:59)
	at org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:128)
	at org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:224)
	at org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)
Caused by: java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]
	at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
	at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
	at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
	at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
	at scala.concurrent.Await$.result(package.scala:107)
	at org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:144)
	at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:60)
	at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:59)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	... 4 more


展开
收起
爱吃鱼的程序员 2020-06-10 10:46:21 801 0
1 条回答
写回答
取消 提交回答
  • https://developer.aliyun.com/profile/5yerqm5bn5yqg?spm=a2c6h.12873639.0.0.6eae304abcjaIB

    用spark-submit提交试试

    应该是本地和spark集群不通吧

    <atarget="_blank"rel="nofollow">https://github.com/jacksu/utils4s 

    已经解决了,sparkdriver的ip映射错了,映射到另一块网卡上了谢谢两位关注<spanstyle="font-size:13.3333px;">请问,你这儿说的<spanstyle="font-family:'MicrosoftYaHei',Verdana,sans-serif,SimSun;font-size:14px;line-height:normal;background-color:#FFFFFF;">sparkdriver的ip映射错了到底是怎么回事,麻烦请告知:601983106@qq.com

    2020-06-10 10:46:39
    赞同 展开评论 打赏
问答排行榜
最热
最新

相关电子书

更多
Hybrid Cloud and Apache Spark 立即下载
Scalable Deep Learning on Spark 立即下载
Comparison of Spark SQL with Hive 立即下载