Scala 在Apache Spark中关闭末尾的流会导致java.io.IOException:流已关闭
不久前,我一直面临着在ApacheSpark中读取zip文件的问题。我已经分享了 正如@Programmer恰当地指出的,我并没有关闭打开的流。我在Scala 在Apache Spark中关闭末尾的流会导致java.io.IOException:流已关闭,scala,apache-spark,stream,inputstream,apache-spark-2.0,Scala,Apache Spark,Stream,Inputstream,Apache Spark 2.0,不久前,我一直面临着在ApacheSpark中读取zip文件的问题。我已经分享了 正如@Programmer恰当地指出的,我并没有关闭打开的流。我在takeWhile() 但它不起作用 在读取zip文件时,我现在遇到以下错误: Job aborted due to stage failure: Task 0 in stage 5.0 failed 1 times, most recent failure: Lost task 0.0 in stage 5.0 (TID 20, localhost
takeWhile
()
但它不起作用强>
在读取zip文件时,我现在遇到以下错误:
Job aborted due to stage failure: Task 0 in stage 5.0 failed 1 times, most recent failure: Lost task 0.0 in stage 5.0 (TID 20, localhost): java.io.IOException: Stream closed
at java.util.zip.ZipInputStream.ensureOpen(ZipInputStream.java:67)
at java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:116)
只要把它打开就行了
因此,我似乎正在关闭流,然后再次尝试读取它。但我不知道为什么以及如何解决它。在@alexandre dupriez评论之后,我只关闭了外部
流,这有助于
Stream.continually(zis.getNextEntry)
.takeWhile {
case null => zis.close(); false
case _ => true
}
.flatMap { _ =>
val br = new BufferedReader(new InputStreamReader(zis))
Stream.continually(br.readLine()).takeWhile(_ != null)
}
因此,我需要一段时间来确认它是否正常工作。在@alexandre dupriez评论之后,我只关闭了外部流,这有助于
Stream.continually(zis.getNextEntry)
.takeWhile {
case null => zis.close(); false
case _ => true
}
.flatMap { _ =>
val br = new BufferedReader(new InputStreamReader(zis))
Stream.continually(br.readLine()).takeWhile(_ != null)
}
因此,我需要一段时间来确认它是否正常工作。这不是因为当您关闭br
流时,您一直在关闭zis
?不是因为当您关闭br
流时,您一直在关闭zis
?