Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Scala 如何声明调用cogroup的函数_Scala_Apache Spark - Fatal编程技术网

Scala 如何声明调用cogroup的函数

Scala 如何声明调用cogroup的函数,scala,apache-spark,Scala,Apache Spark,我想声明一个函数来获取两个RDD的cogroup。实际上这是一个交叉键。无法编译以下代码: def getRetain[K, V](activeUserRdd : RDD[(K, V)], newUserRdd : RDD[(K, V)]): RDD[(K, V)] ={ activeUserRdd.cogroup(newUserRdd).flatMapValues{ x => Option((if (!x._1.isEmpty && !x._2.isE

我想声明一个函数来获取两个RDD的cogroup。实际上这是一个交叉键。无法编译以下代码:

def getRetain[K, V](activeUserRdd : RDD[(K, V)], newUserRdd : RDD[(K, V)]): RDD[(K, V)] ={
    activeUserRdd.cogroup(newUserRdd).flatMapValues{
      x => Option((if (!x._1.isEmpty && !x._2.isEmpty) x._2.head else  null).asInstanceOf[V])
    }
  }
错误:

value cogroup is not a member of org.apache.spark.rdd.RDD[(K, V)]
我认为K,V miss与cogroup中声明的实[K,V]匹配,但在我的函数中声明的正确方法是什么?

将类标记应用于输入类型,以确保在运行时可以访问已擦除的类型K和V。这是由于


我没有这个包scala.reflect.TypeTag,你知道其他一些解决方案只是使用原始的scala。我想了解scala对我有很大帮助。编译时抛出了错误。谢谢。对不起,我输入错了。请再试一次。
scala> import scala.reflect.ClassTag
import scala.reflect.ClassTag

scala> def getRetain[K : ClassTag, V : ClassTag](activeUserRdd : RDD[(K, V)], newUserRdd : RDD[(K, V)]): RDD[(K, V)] ={
 |       activeUserRdd.cogroup(newUserRdd).flatMapValues{
 |         x => Option((if (!x._1.isEmpty && !x._2.isEmpty) x._2.head else  null).asInstanceOf[V])
 |       }
 |     }
 getRetain: [K, V](activeUserRdd: org.apache.spark.rdd.RDD[(K, V)], newUserRdd: org.apache.spark.rdd.RDD[(K, V)])(implicit evidence$1: scala.reflect.ClassTag[K], implicit evidence$2: scala.reflect.ClassTag[V])org.apache.spark.rdd.RDD[(K, V)]