Caused by: java.io.FileNotFoundException: File file:/tmp/spark-events does not exist
Exception in thread "main" java.io.FileNotFoundException: Log directory specified does not exist: file:/tmp/spark-events Did you configure the correct one through spark.history.fs.logDirectory? at org.apache.spark.deploy.history.FsHistoryProvider.startPolling(FsHistoryProvider.scala:279) at org.apache.spark.deploy.history.FsHistoryProvider.initialize(FsHistoryProvider.scala:227) at org.apache.spark.deploy.history.FsHistoryProvider.start(FsHistoryProvider.scala:409) at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:303) at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala) Caused by: java.io.FileNotFoundException: File file:/tmp/spark-events does not exist at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:666) at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:987) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:656) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:454) at org.apache.spark.deploy.history.FsHistoryProvider.startPolling(FsHistoryProvider.scala:269) ... 4 more
当我们配置Spark历史服务,当使用命令sbin/start-history-server.sh进行启动时,如果报上述错误,是因为在/tmp中没有spark-events文件夹
直接在/tmp路径创建该文件夹就可以了