Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark 如何重用spark stream的结果?_Apache Spark_Spark Streaming - Fatal编程技术网

Apache spark 如何重用spark stream的结果?

Apache spark 如何重用spark stream的结果?,apache-spark,spark-streaming,Apache Spark,Spark Streaming,我们如何使用地图中的值,似乎这些值没有被填充 val goalScore = rawScore.transform(rdd=>{ val minMax = rdd.flatMap(x=>{ x.behaviorProfileType match { case Some("mapper") => Some((x.sourceType, x.targetType, "mapper"), x)

我们如何使用地图中的值,似乎这些值没有被填充

  val goalScore = rawScore.transform(rdd=>{
        val minMax = rdd.flatMap(x=>{
          x.behaviorProfileType match {
            case Some("mapper") => Some((x.sourceType, x.targetType, "mapper"), x)          
            case Some("non-mappe") => Some((x.sourceType, x.targetType, "non-mapper"), x)
            case _ => None     
          }
        })
        .reduceByKey(reduceMinMax(_, _))
        .collectAsMap()

        rdd.map(x => (populateMinMaxWindowGoalScore(x, minMax)))
      })

为什么在populateMinMaxWindowGoalScore函数中minMax始终为空?rawScore是一个数据流。

您能扩展您的代码吗?另外,如何调试minmax为空?输入是什么?请提供更多详细信息。您是否可以扩展代码?另外,如何调试minmax为空?输入是什么?请提供更多详情