Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Scala 如何转换Avro的RDD';s GenericData。是否记录到数据帧?_Scala_Apache Spark_Apache Spark Sql_Avro - Fatal编程技术网

Scala 如何转换Avro的RDD';s GenericData。是否记录到数据帧?

Scala 如何转换Avro的RDD';s GenericData。是否记录到数据帧?,scala,apache-spark,apache-spark-sql,avro,Scala,Apache Spark,Apache Spark Sql,Avro,也许这个问题看起来有点抽象,这里是: val originalAvroSchema : Schema = // read from a file val rdd : RDD[GenericData.Record] = // From some streaming source // Looking for a handy: val df: DataFrame = rdd.toDF(schema) 我进行了探索,但它只支持从文件中读取,而不支持从现有的RDD中读取。我不确定为什么删除了

也许这个问题看起来有点抽象,这里是:

val originalAvroSchema : Schema   = // read from a file
val rdd : RDD[GenericData.Record] = // From some streaming source

// Looking for a handy:
val df: DataFrame   = rdd.toDF(schema)

我进行了探索,但它只支持从文件中读取,而不支持从现有的
RDD

中读取。我不确定为什么删除了答案。这里有一个对您要查找的内容的请求。我删除了答案,因为它已被否决。
import com.databricks.spark.avro._

val sqlContext = new SQLContext(sc)
val rdd : RDD[MyAvroRecord] = ...
val df = rdd.toAvroDF(sqlContext)