Scala 键值对映射-从数据帧到cassandra表
json解析数据帧(insertDF)的格式如下:Scala 键值对映射-从数据帧到cassandra表,scala,apache-spark,cassandra,Scala,Apache Spark,Cassandra,json解析数据帧(insertDF)的格式如下: +------------------------------------------------+-------------+------------+ Cls_details | prdct_cde |roll_nbr +------------------------------------------------+-------------+--------
+------------------------------------------------+-------------+------------+
Cls_details | prdct_cde |roll_nbr
+------------------------------------------------+-------------+------------+
{"key1":"value1","key2":"value2","key3":"value3"}|DKLA |123453
{"key1":"value1","key2":"value2","key3":"value3"}|GHTD |123454
{"key1":"value1","key2":"value2","key3":"value3"}|ILDDA |123455
我想将此数据帧保存到cassandra表中。
使用的spark scala代码是:
insertDF
.write.format("org.apache.spark.sql.cassandra")
.options(Map("keyspace" -> "abc", "table" -> "def"))
.mode(SaveMode.Append).save()
但我发现了一个错误:
com.datastax.spark.connector.types.TypeConversionException: Cannot convert object {"key1":"value1","key2":"value2","key3":"value3"} of type class java.lang.String to Map[AnyRef,AnyRef].
请在spark scala中建议解决方案。
非常感谢您的帮助。您需要将Cls\U详细信息转换为地图类型。。我试过了。它不起作用。您能否建议将struct类型的架构更改为map的代码。在Cassandra中,列类型是冻结的。您需要将Cls_详细信息转换为映射类型。。我试过了。它不起作用。您能否建议将struct类型的架构更改为map的代码。在Cassandra中,列类型是冻结的