Scala 将行值从Spark df传递给函数
我需要将每一行及其coresponding值传递给上述函数。 第一行中数字列的值=8传递给numbervalidation,第一行中的word列abc传递给WordValidationin使用spark并按如下方式更改您的方法:Scala 将行值从Spark df传递给函数,scala,apache-spark,apache-spark-sql,Scala,Apache Spark,Apache Spark Sql,我需要将每一行及其coresponding值传递给上述函数。 第一行中数字列的值=8传递给numbervalidation,第一行中的word列abc传递给WordValidationin使用spark并按如下方式更改您的方法: val someDF = Seq( (8, "abc"), (64, "sdf"), (-27, "yui") ).toDF("number", "word"
val someDF = Seq(
(8, "abc"),
(64, "sdf"),
(-27, "yui")
).toDF("number", "word")
def numberValidation(numberValidation:Any): Boolean=
numberValidation match{
case int :Integer => true
case _ => false}
def wordValidatoin(wordValidation:Any): Boolean=
wordValidation match{
case str: String => true
case _ => false}
val numberValidation = udf((numberValidation: Any) => {
numberValidation match{
case int :Integer => true
case _ => false
}
})
val wordValidatoin = udf((wordValidation:Any)): =>{
wordValidation match{
case str: String => true
case _ => false
}
})
并与DF进行如下检查:
val someDF = Seq(
(8, "abc"),
(64, "sdf"),
(-27, "yui")
).toDF("number", "word")
def numberValidation(numberValidation:Any): Boolean=
numberValidation match{
case int :Integer => true
case _ => false}
def wordValidatoin(wordValidation:Any): Boolean=
wordValidation match{
case str: String => true
case _ => false}
val numberValidation = udf((numberValidation: Any) => {
numberValidation match{
case int :Integer => true
case _ => false
}
})
val wordValidatoin = udf((wordValidation:Any)): =>{
wordValidation match{
case str: String => true
case _ => false
}
})