Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
不兼容类型使用转换将JavaPairRDD连接到JavaPairDStream时出错_Java_Apache Spark_Spark Streaming - Fatal编程技术网

不兼容类型使用转换将JavaPairRDD连接到JavaPairDStream时出错

不兼容类型使用转换将JavaPairRDD连接到JavaPairDStream时出错,java,apache-spark,spark-streaming,Java,Apache Spark,Spark Streaming,我试图使用transform将JavaPairRDD连接到JavaPairDStream中的RDD,但出现以下错误: incompatible types: no instance(s) of type variable(s) W exist so that org.apache.spark.api.java.JavaPairRDD<java.lang.String,scala.Tuple2<LogType,W>> conforms to org.apache.spark

我试图使用transform将JavaPairRDD连接到JavaPairDStream中的RDD,但出现以下错误:

incompatible types: no instance(s) of type variable(s) W exist so that org.apache.spark.api.java.JavaPairRDD<java.lang.String,scala.Tuple2<LogType,W>> conforms to org.apache.spark.api.java.JavaRDD<U>
不兼容类型:不存在类型变量W的实例,因此org.apache.spark.api.java.javapairdd符合org.apache.spark.api.java.JavaRDD
这是我的密码:

// Load full table from C*
JavaPairRDD<String,CassTableType> table = javaFunctions(
  jssc
).cassandraTable(
  "mykeyspace", "mytable", mapRowTo(CassTableType.class)
).keyBy(
  t -> t.getLogId()
);

// Process stream
JavaPairDStream<String,LogType> logs = flumeStream.flatMapToPair(
  flumeEvent -> { 
    List<Tuple2<String, LogType>> events = new LinkedList<>();
    LogType log = LogType.parse(flumeEvent);
    events.add(new Tuple2<String, LogType>(log.getLogID(), log));
    return events;
  }
);

// Join RDD's in logs DStream to table
JavaPairDStream<String, Tuple2<LogType, CassTableType>> 
  joinedRDD = logs.transform(
    rdd -> rdd.join(table))
);
//从C加载完整的表*
javapairdd表=javaFunctions(
jssc
).卡桑德拉(
“mykeyspace”、“mytable”、mapRowTo(casstTableType.class)
)凯比先生(
t->t.getLogId()
);
//工艺流程
JavaPairDStream日志=flumeStream.flatMapToPair(
flumeEvent->{
列表事件=新建LinkedList();
LogType log=LogType.parse(flumeEvent);
添加(新的Tuple2(log.getLogID(),log));
返回事件;
}
);
//将日志数据流中的RDD连接到表
JavaPairDStream
joinedd=logs.transform(
rdd->rdd.join(表))
);

这是否意味着rdd是
JavaRDD
而不是
javapairdd
根据我正在转换的
JavaPairDStream
?如何实现此连接?

使用transformToPair而不是transform解决了此问题