Apache kafka Kafka中KSQL操作的主题数据格式

Apache kafka Kafka中KSQL操作的主题数据格式,apache-kafka,confluent-platform,ksqldb,Apache Kafka,Confluent Platform,Ksqldb,我刚开始使用ksql,当我从一开始就打印主题时,我得到了以下格式的数据 rowtime: 4/12/20, 9:00:05 AM MDT, key: {"messageId":null}, value: {"WHS":[{"Character Set":"UTF-8","action":"finished","Update-Date-Time":"2020-04-11 09:00:02:25","Number":0,"Abbr":"","Name":"","Name2":"","Count

我刚开始使用ksql,当我从一开始就打印主题时,我得到了以下格式的数据

   rowtime: 4/12/20, 9:00:05 AM MDT, key: {"messageId":null}, value: {"WHS":[{"Character Set":"UTF-8","action":"finished","Update-Date-Time":"2020-04-11 09:00:02:25","Number":0,"Abbr":"","Name":"","Name2":"","Country-Code":"","Addr-1":"","Addr-2":"","Addr-3":"","Addr-4":"","City":"","State":""}]}
但是KSQL中的所有示例都有以下格式的数据

{"ROWTIME":1537436551210,"ROWKEY":"3375","rating_id":3375,"user_id":2,"stars":3,"route_id":6972,"rating_time":1537436551210,"channel":"web","message":"airport refurb looks great, will fly outta here more!"}
所以我无法执行任何操作,格式显示为

Key format: JSON or SESSION(KAFKA_STRING) or HOPPING(KAFKA_STRING) or TUMBLING(KAFKA_STRING) or KAFKA_STRING
Value format: JSON or KAFKA_STRING
关于我的话题。如何将数据修改为特定格式


谢谢,ksqlDB还不支持JSON消息键(请参阅跟踪)

但是,您仍然可以访问键和值中的数据。JSON键毕竟只是一个字符串

重新格式化后的值如下所示:

{
  "WHS":[
    {
      "Character Set":"UTF-8",
      "action":"finished",
      "Update-Date-Time":"2020-04-11 09:00:02:25",
      "Number":0,
      "Abbr":"",
      "Name":"",
      "Name2":"", 
      "Country-Code":"",
      "Addr-1":"",
      "Addr-2":"",
      "Addr-3":"",
      "Addr-4":"",
      "City":"",
      "State":""
    }
  ]
}
假设所有行共享一个公共格式,ksqlDB可以轻松地处理

要导入流,您应该能够运行如下操作:

{
  "WHS":[
    {
      "Character Set":"UTF-8",
      "action":"finished",
      "Update-Date-Time":"2020-04-11 09:00:02:25",
      "Number":0,
      "Abbr":"",
      "Name":"",
      "Name2":"", 
      "Country-Code":"",
      "Addr-1":"",
      "Addr-2":"",
      "Addr-3":"",
      "Addr-4":"",
      "City":"",
      "State":""
    }
  ]
}
——假设卡夫卡的v0.9
创建流内容
(
行键字符串键,
WHS阵列<
结构<
`字符集`字符串,
动作串,
`更新日期时间字符串,
数字字符串,
等
>
>
)
使用(kafka_topic='?',value_format='JSON');

值列
WHS
是一个结构数组(其中只有一个元素),结构定义了需要访问的所有字段。请注意,有些字段名需要引用,因为它们包含无效字符,例如空格和破折号。

我尝试以Avro格式创建Value_格式的流,但仍然不起作用。