开发者社区> 问答> 正文

通过`in-code variable inspection`调试scala中的过滤器操作[重复]

社区小助手 2018-12-21 13:41:13 265

def main(args:Array[String]){

Logger.getLogger("org").setLevel(Level.ERROR)
val sc = new SparkContext("local[*]","WordCountRe")
val input = sc.textFile("data/book.txt")
//With regexp
val words = input.flatMap(x=>x.split("\\W+"))
//Lower case
val lowerCaseWords = words.map(x => x.toLowerCase())
val wordCounts = lowerCaseWords.map(x => (x,1)).reduceByKey((x,y)=>x+y)
val sortedWordCounts = wordCounts.sortBy(-_._2)
val commonEnglishStopWords = List("you","to","your","the","a","of","and","that","it","in","is","for","on","are","if","s","i","with","t","this","or","but","they","will","what","at","my","re","do","not","about","more","an","up","need","them","from","how","there","out","new","work","so","just","don","","get","their","by","some","ll","self","make","may","even","when","one","than","also","much","job","who","was","these","find","into","only")
val filteredWordCounts = sortedWordCounts.filter{
  x =>
    val inspectVariable = commonEnglishStopWords.contains(x._1)} //Error here
filteredWordCounts.collect().foreach(println)   } }

当我尝试使用此代码时,出现编译错误:

类型不匹配; 发现:需要的单位:布尔WordCountRe.scala / SparkScalaCourse / src / com / sundogsoftware / spark line 29 Scala问题

发现我的代码出了什么问题(._1为了解析元组中的单词(word,count)需要放入包含),但我仍然不知道在这种情况下如何调试/检查值。

分享到
取消 提交回答
全部回答(1)
  • 社区小助手
    2019-07-17 23:23:23

    问题是你将方法的布尔结果赋给contains了val inspectVariable。此操作的返回类型为Unit。但filter方法需要布尔值。

    只需删除val inspectVariable =,这应该解决它。

    或者inspectVariable在分配值后通过添加包含内容的新行来返回值。

    如图所示

    val filteredWordCounts = sortedWordCounts.filter { x =>
    val inspectVariable = commonEnglishStopWords.contains(x._1)//put your breakpoint here
    inspectVariable
    }

    0 0