Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/scala/16.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Scala 错误:值更新不是org.apache.spark.sql.DataFrame的成员_Scala_Apache Spark - Fatal编程技术网

Scala 错误:值更新不是org.apache.spark.sql.DataFrame的成员

Scala 错误:值更新不是org.apache.spark.sql.DataFrame的成员,scala,apache-spark,Scala,Apache Spark,我有下面的代码抛出一个错误,如下所示。无法解决这个问题 val someDF = Seq((8, "bat"),(64, "mouse"),(-27, "horse")).toDF("number", "word") val someDF2 = Seq((8, "bat"),(64, "cat"),(-45, "tiger")).toDF("number", "word") val someDF1= someDF.withColumn("Action",lit("A")) val max_row

我有下面的代码抛出一个错误,如下所示。无法解决这个问题

val someDF = Seq((8, "bat"),(64, "mouse"),(-27, "horse")).toDF("number", "word")
val someDF2 = Seq((8, "bat"),(64, "cat"),(-45, "tiger")).toDF("number", "word")
val someDF1= someDF.withColumn("Action",lit("A"))
val max_rows= someDF1.count().toInt
for(i <- 0 to max_rows) {
  if (someDF1("number") === someDF2("number")) {
    if (someDF1("word") === someDF2("word")) {
      someDF1("Action") = "D"
    } else {
      someDF1.unionAll(someDF2.select($"*", lit("D")))
    }
  }
}

错误消息可能有点混乱,因为
update
是由
someDF1(“Action”)=“D”
(相当于
someDF1.update(“Action”,“D”)
产生的desugared方法。正如错误消息所说,没有
更新
,因为数据集是不可变的。重复:特别是中的第四个要点。谢谢@AndreyTyukin。我将研究它而不是循环,我使用了一个适合我的联接。。val-cpdf=Seq((8,“bat”),(64,“mouse”),(-27,“horse”).toDF(“number”,“word”)val-tabdf=Seq((8,“bat”),(64,“cat”),(-45,“tiger”).toDF(“no”,“word”)val-cpdf1=cpdf.withColumn(“Action”,lit(“A”))val-groupedData\u col=cpdf1.as(“cpdf1”)。join(tabdf.as(“tabdf”)、(col.word”)、(cpdf.word”)==col)col=“col=“col.word”)、”col==col==col.no”)、(“tabdf,“left_outer”).filter(col(“tabdf.word”)为null)。选择(col(“cpdf1.*)错误消息可能会有点混淆,因为
update
是由
someDF1(“Action”)=“D”
”产生的desugared方法,相当于
someDF1.update(“Action”,“D”)
。正如错误消息所说,没有更新,因为数据集是不可变的。在副本中:特别是中的第四个项目符号。感谢@AndreyTyukin。将深入研究它而不是循环,我使用了一个对我有效的连接。..val cpdf=Seq((8,“bat”),(64,“mouse”),(.27,“horse”)。toDF(“number”,“word”)val tabdf=Seq((8,“bat”),(64,“cat”),(-45,“tiger”).toDF(“no”,“word”)val cpdf1=cpdf.withColumn(“Action”,lit(“A”))val groupedData\u col=cpdf1.as(“cpdf1”)。join(tabdf.as(“tabdf”)、col(“cpdf1.word”)==col(“tabdf.word”)和col(“cpdf1.number”)==col(“tabdf.no”)、“left\u outer”)。过滤器(col(“tabdf.word”)为空。选择(col)(“cpdf1”。)
              someDF1("Action") = "D"
                                ^