Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark Spark streaming NetworkWordCount示例每批创建多个作业_Apache Spark_Spark Streaming - Fatal编程技术网

Apache spark Spark streaming NetworkWordCount示例每批创建多个作业

Apache spark Spark streaming NetworkWordCount示例每批创建多个作业,apache-spark,spark-streaming,Apache Spark,Spark Streaming,我正在通过spark shell在纱线簇上运行基本的NetworkWordCount程序。这是我的代码片段- import org.apache.spark.streaming.{Seconds, StreamingContext} import org.apache.spark.storage.StorageLevel val ssc = new StreamingContext(sc, Seconds(60)) val lines = ssc.socketTextStream("172.2

我正在通过spark shell在纱线簇上运行基本的NetworkWordCount程序。这是我的代码片段-

import org.apache.spark.streaming.{Seconds, StreamingContext}
import org.apache.spark.storage.StorageLevel

val ssc = new StreamingContext(sc, Seconds(60))
val lines = ssc.socketTextStream("172.26.32.34", 9999, StorageLevel.MEMORY_ONLY)
val words = lines.flatMap(_.split(" "))
val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _)
wordCounts.print()
ssc.start()
ssc.awaitTermination()
console上的输出和Streaming选项卡上的stats也与预期一样。 但当我查看“作业”选项卡时,每1分钟批处理间隔触发两个作业,不应该是每间隔触发一个作业吗?下面是截图-

现在,当我在流媒体UI上查看已完成的批次时,我看到每分钟只有一个批次-

我错过什么了吗?另外,我注意到开始作业也有两个同名的状态,它们产生了不同数量的任务,如下图所示,这里到底发生了什么

相关问题(不幸也未回答)相关问题(不幸也未回答)