Scala 错误:值cassandraFormat不是org.apache.spark.sql.DataFrameWriter的成员
检查github上的回购我看到了Scala 错误:值cassandraFormat不是org.apache.spark.sql.DataFrameWriter的成员,scala,apache-spark,apache-spark-sql,spark-cassandra-connector,Scala,Apache Spark,Apache Spark Sql,Spark Cassandra Connector,检查github上的回购我看到了cassandraFormat。我的导入语句未引发异常: import org.apache.spark.sql.cassandra._ df.write .cassandraFormat("keyspace", "table") .save() <console>:34: error: value cassandraFormat is not a member of org.apache.spark.sql.DataFrameWr
cassandraFormat
。我的导入语句未引发异常:
import org.apache.spark.sql.cassandra._
df.write
.cassandraFormat("keyspace", "table")
.save()
<console>:34: error: value cassandraFormat is not a member of org.apache.spark.sql.DataFrameWriter[org.apache.spark.sql.Row]
编辑: 我确实意识到,
cassandraFormat
基本上是.format().options()
的别名。但是,返回了一个不同的错误:
df.write
.format("org.apache.spark.sql.cassandra")
.options(Map("table" -> "standard_feed", "keyspace" -> "testing"))
.save()
java.lang.AbstractMethodError: org.apache.spark.sql.cassandra.DefaultSource.createRelation
(Lorg/apache/spark/sql/SQLContext;Lorg/apache/spark/sql/SaveMode;Lscala/collection/
immutable/Map;Lorg/apache/spark/sql/Dataset;)Lorg/apache/spark/sql/sources/
BaseRelation;
at org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:518)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:215)
我对cassandraFormat不太确定,但将数据帧保存到cassandra表可以通过以下方式最简单地完成:
df.write
.format("org.apache.spark.sql.cassandra")
.options(Map("table" -> "table_name", "keyspace" -> "keyspace_name"))
.save()
关于您收到的错误消息:
我认为您收到的错误消息是由于版本不匹配造成的。您使用spark 2.1.1和cassandra连接器版本1.6.6。对于spark 2.1.x,您需要使用2.0版的cassandra连接器,有关版本兼容性的完整列表,请参阅表
df.write
.format("org.apache.spark.sql.cassandra")
.options(Map("table" -> "table_name", "keyspace" -> "keyspace_name"))
.save()