Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark 将spark行转换为case类,如果值为null,则转换失败_Apache Spark - Fatal编程技术网

Apache spark 将spark行转换为case类,如果值为null,则转换失败

Apache spark 将spark行转换为case类,如果值为null,则转换失败,apache-spark,Apache Spark,我想将数据帧转换为数据集[CC] case class CC(id: String, value: String) df.as[CC] 但是,如果id或值中的值为null,我希望该操作引发异常。这可能吗 也许这样: val ds = df.map(s=>{ s.toSeq.foreach(x=>{ if (x == null) throw new Exception("null value: "+s.mkString(",")) }) new

我想将数据帧转换为数据集[CC]

case class CC(id: String, value: String)
df.as[CC]
但是,如果id或值中的值为null,我希望该操作引发异常。这可能吗

也许这样:

 val ds = df.map(s=>{
    s.toSeq.foreach(x=>{
      if (x == null) throw new Exception("null value: "+s.mkString(","))
    })
    new CC(s.getString(0),s.getString(1))
  })
例外情况:

`Caused by: java.lang.Exception: null value: 2,null`
`Caused by: java.lang.Exception: null value: 2,null`