如何在scala中减少来自kafka的数据流数据?

如何在scala中减少来自kafka的数据流数据?,scala,Scala,我有一些来自kafka流媒体的数据流,我想使用scala以所需的格式减少数据流。 请为我提供提示,以获取给定数据流数据的正确格式 例如: case class Test(cust: String, labNo: String, device: String, properties: String, maxValue: Int, count: Int)] DStream[((cust_1,lab2, switch,cpu,90,2), (cust_1,lab2, switch,

我有一些来自kafka流媒体的数据流,我想使用scala以所需的格式减少数据流。 请为我提供提示,以获取给定数据流数据的正确格式

例如:

case class Test(cust: String, labNo: String, device: String, properties: String, maxValue: Int, count: Int)]

DStream[((cust_1,lab2, switch,cpu,90,2),
         (cust_1,lab2, switch,cpu,80,4),
         (cust_1,lab2, ap,cpu,70,4),
         (cust_1,lab2, switch,mem,70,3),
         (cust_1,lab1, switch,cpu,90,2),
         (cust_1,lab1, switch,cpu,70,4),
         (cust_1,lab1, switch,cpu,80,4),
         (cust_1,lab1, ap,mem,70,3))]
格式化(k,v)并应用 Groupbykey(客户、实验室)=>

获得以下预期结果:

(cust_1, lab2), ArrayBuffer((switch, cpu, 90, 2), (switch,cpu,80,4), (ap,cpu,70,4), (switch,mem,70,3))
(cust_1, lab1), ArrayBuffer((switch, cpu 90, 2), (switch,cpu,70,4), (switch,cpu,80,4), (ap,mem,70,3))
阵列缓冲区内的GroupbyKey(设备):

(cust_1, lab12), ArrayBuffer((switch -> ((cpu, 90, 2), (cpu,80,4),(mem,70,3)), ap -> (cpu,70,4)))
(cust_1, lab1), ArrayBuffer((switch -> ((cpu 90, 2), (cpu,70,4), (cpu,80,4)), ap->(mem,70,3))
每个设备的GroupbyKey(属性):

(cust_1, lab12), ArrayBuffer((switch -> (cpu -> ((90, 2), (80,4)), mem ->(70,3))), ap -> (cpu,70,4)))
(cust_1, lab1), ArrayBuffer((switch -> (cpu -> ((90, 2), (70,4), (80,4)))), ap->(mem,70,3)))
我用上面的例子来解释这个要求。该示例未执行,因此请忽略任何Scala输出格式

因此,最终输出如下所示:

result {
   cust: "xyz"
   labNo: 2
   device {
      switch {
             cpu {
                 maxValue: 80
                 count: 4
                 },
             cpu {
                 maxValue: 90
                 count: 2
                 }
              mem {
                maxValue: 70
                count: 3
                }
      }
      ap {
             cpu {
                 maxValue: 70
                 count: 4
                 }
              mem {
                maxValue: 70
                count: 3
                }
      }

   }
}