Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/three.js/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache flink 在Flink中使用定义的聚合函数-未找到函数签名的匹配项_Apache Flink_Flink Streaming_Flink Sql_Flink Batch - Fatal编程技术网

Apache flink 在Flink中使用定义的聚合函数-未找到函数签名的匹配项

Apache flink 在Flink中使用定义的聚合函数-未找到函数签名的匹配项,apache-flink,flink-streaming,flink-sql,flink-batch,Apache Flink,Flink Streaming,Flink Sql,Flink Batch,我想保留Flink中“Select..From..groupby..”查询中每个键的所有原始行。我定义了一个名为RowToJsonAgg的AggregateFunction,它将行聚合成一个Json字符串 class RowToJsonAgg extends AggregateFunction[String, ListBuffer[String]]{ def accumulate(accumulator: ListBuffer[String], row: Any*): Unit = {

我想保留Flink中“Select..From..groupby..”查询中每个键的所有原始行。我定义了一个名为RowToJsonAgg的AggregateFunction,它将行聚合成一个Json字符串

class RowToJsonAgg extends AggregateFunction[String, ListBuffer[String]]{
  def accumulate(accumulator: ListBuffer[String], row: Any*): Unit = {
   ....
//假设行看起来像“$field1\u name,$field1\u value,$field2\u name,$field2\u value,…” //尝试从行生成json。然而,当我运行查询时,Flink似乎找不到这个函数 }

数据类和查询如下所示:

case class Stock(id:Int, price: Int, volumn: Int, ts: Long)

val bbSettings = EnvironmentSettings.newInstance().useBlinkPlanner().inBatchMode().build()

val bbTableEnv = TableEnvironment.create(bbSettings)

bbTableEnv.createTemporarySystemFunction("row_to_json_agg", classOf[RowToJsonAgg])

val table = bbTableEnv.fromValues(...)

bbTableEnv.createTemporaryView("Stock", table)

bbTableEnv.executeSql(
    "select price, row_to_json_agg('volumn', volumn, 'ts', ts) as details from Stock group by price"
)
当我运行应用程序时,我得到了SQL验证异常,详细消息是“未找到函数签名行\u到\u json\u agg(字符、数字、字符、数字)的匹配项”

似乎Flink找不到要调用的正确的累计函数

如果我按如下方式声明累积函数

def accumulate(accumulator: ListBuffer[String], volumn: Integer, ts: Long)
并将查询更改为

"select price, row_to_json_agg(volumn, ts) from Stock group by price" 
我得到了相同的异常,消息是“未找到函数签名行\u到\u json\u agg(数字,数字)


有什么办法让聚合函数工作吗?

我自己想出来的

  • 通过如下方式运行SQL来注册UDF:

    case class Stock(id:Int, price: Int, volumn: Int, ts: Long)
    
    val bbSettings = EnvironmentSettings.newInstance().useBlinkPlanner().inBatchMode().build()
    
    val bbTableEnv = TableEnvironment.create(bbSettings)
    
    bbTableEnv.createTemporarySystemFunction("row_to_json_agg", classOf[RowToJsonAgg])
    
    val table = bbTableEnv.fromValues(...)
    
    bbTableEnv.createTemporaryView("Stock", table)
    
    bbTableEnv.executeSql(
        "select price, row_to_json_agg('volumn', volumn, 'ts', ts) as details from Stock group by price"
    )
    
    bbTableEnv.executeSQL( format(“将临时函数$udf\U名称创建为“%s”,$full\U class\U name\U of_your\U udf”) )

  • 而不是

    bbTableEnv.createTemporarySystemFunction("row_to_json_agg", classOf[RowToJsonAgg])
    
  • 宁愿使用Java来实现UDF而不是Scala