Job failed with org.apache.spark.SparkException: Job aborted due to stage failure: Task 3 in stage 0.0 failed 4 times, most recent failure: Lost task 3.3 in stage 0.0 (TID 14, slave5, executor 4): ExecutorLostFailure (executor 4 exited caused by one of the running tasks) Reason: Unable to create executor due to Unable to register with external shuffle server due to : java.lang.IllegalStateException: Expected SaslMessage, received something else (maybe your client does not have SASL enabled?)
问题
如何将hive集群升级到spark,hivesql脚本运行比较慢
参数设置 set hive.execution.engine=spark; 之后报错Expected SaslMessage, received something else (maybe your client does not have SASL enabled
思路
spark.authenticate=false
把SASL 认证关了
spark.executor.cores=6 //根据实际内核数设置提升cpu使用率
解决
成功解决
