Java ApacheKafka-实现KTable

Java ApacheKafka-实现KTable,java,apache-kafka,apache-kafka-streams,ktable,Java,Apache Kafka,Apache Kafka Streams,Ktable,我是Kafka Streams API的新手,我正在尝试创建一个KTable。我有一个输入主题:s-order-topic,它是一个json格式的消息,如下所示 { "current_ts": "2019-12-24 13:16:40.316952", "primary_keys": ["ID"], "before": null, "tokens": {"txid":"3.17.2493", "csn":"64913009"}, "op_type":"

我是Kafka Streams API的新手,我正在尝试创建一个KTable。我有一个输入主题:
s-order-topic
,它是一个json格式的消息,如下所示

{ "current_ts": "2019-12-24 13:16:40.316952",
  "primary_keys": ["ID"],
  "before": null,
  "tokens": {"txid":"3.17.2493", 
             "csn":"64913009"},
  "op_type":"I",
  "after":  { "CODE":"AAAA41",
              "STATUS":"COMPLETED",
              "ID":24},
  "op_ts":"2019-12-24 13:16:40.316941",
  "table":"S_ORDER"} 
我阅读了本主题中的消息,我想创建一个KTable,该表包含as键、字段
“之后”:“ID”
,以及字段(
“ID”
)之后
)中的所有字段(除了
“ID”

只有在使用默认聚合函数(即count)时,我才成功创建了KTable。但是我很难创建自己的聚合函数。下面我展示了我试图创建KTable的代码部分

KTable<Long, String> s_table = builder.stream("s-order-topic",  Consumed.with(Serdes.Long(),Serdes.String()))
                .mapValues(value -> {
                    String time;
                    JSONObject json = new JSONObject(value);
                    if (json.getString("op_type").equals("I")) {
                        time = "after";
                    }else {
                        time = "before";
                    }
                    JSONObject json2 = new JSONObject(json.getJSONObject(time).toString());
                    return json2.toString();
                })
               .groupBy((key, value) -> {
                    JSONObject json = new JSONObject(value);
                    return json.getLong("ID");
                }, Grouped.with(Serdes.Long(), Serdes.String()))
                .aggregate( ... );
KTable s_table=builder.stream(“s-order-topic”,consumered.with(Serdes.Long(),Serdes.String())
.mapValues(值->{
串时间;
JSONObject json=新的JSONObject(值);
if(json.getString(“op_type”).equals(“I”)){
time=“after”;
}否则{
time=“before”;
}
JSONObject json2=新的JSONObject(json.getJSONObject(time.toString());
返回json2.toString();
})
.groupBy((键、值)->{
JSONObject json=新的JSONObject(值);
返回json.getLong(“ID”);
},Grouped.with(Serdes.Long(),Serdes.String())
.合计(……);
如何实现这个KTable

我处理问题的方法正确吗


(mapValues->只保留“before”/“after”字段。groupBy->将ID作为消息的键。Aggregate->?)

我想出了一个解决方案。我实现了KTable,如下所示:

 KTable<String, String> s_table = builder.stream("s-order-topic",  Consumed.with(Serdes.String(),Serdes.String()))
                .mapValues(value -> {
                    String time;
                    JSONObject json = new JSONObject(value);
                    if (json.getString("op_type").equals("I")) {
                        time = "after";
                    }else {
                        time = "before";
                    }
                    JSONObject json2 = new JSONObject(json.getJSONObject(time).toString());
                    return json2.toString();
                })
                .groupBy((key, value) -> {
                    JSONObject json = new JSONObject(value);
                    return String.valueOf(json.getLong("ID"));
                }, Grouped.with(Serdes.String(), Serdes.String()))
                .reduce((prev,newval)->newval);
15   {"CODE":"AAAA17","STATUS":"PENDING","ID":15}
18   {"CODE":"AAAA50","STATUS":"SUBMITTED","ID":18}
4    {"CODE":"AAAA80","STATUS":"SUBMITTED","ID":4}
19   {"CODE":"AAAA83","STATUS":"SUBMITTED","ID":19}
18   {"CODE":"AAAA33","STATUS":"COMPLETED","ID":18}
5    {"CODE":"AAAA38","STATUS":"PENDING","ID":5}
10   {"CODE":"AAAA1","STATUS":"COMPLETED","ID":10}
3    {"CODE":"AAAA68","STATUS":"NOT COMPLETED","ID":3}
9    {"CODE":"AAAA89","STATUS":"PENDING","ID":9}

只是一个旁注:聚合是
reduce()
的泛化,允许您更改输出类型(对于reduce,输入值类型==输出值类型)。因此,使用聚合实际上也应该起作用。@MatthiasJ.Sax感谢您的更正!因此,我的问题应该是,我没有为我的案例正确地实现聚合。