Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/xpath/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Scala Kafka Streams PAPI:处理器关闭在启动时被调用,而且调用频率太高_Scala_Apache Kafka_Apache Kafka Streams - Fatal编程技术网

Scala Kafka Streams PAPI:处理器关闭在启动时被调用,而且调用频率太高

Scala Kafka Streams PAPI:处理器关闭在启动时被调用,而且调用频率太高,scala,apache-kafka,apache-kafka-streams,Scala,Apache Kafka,Apache Kafka Streams,如中所示,您应该关闭处理器中使用的存储,并覆盖close方法 在WordCountProcessor示例中,它显示了如何在调用close()方法中关闭存储 我也做过类似的事情(我没有在init()方法中启动它们,而是在Scala中使用lazy val),我发现我的Processor close()方法在创建存储之后被多次调用 class EventWindowProcessor(sessionStoreName: String, lastSessionByChannelStoreName: St

如中所示,您应该关闭处理器中使用的存储,并覆盖close方法

在WordCountProcessor示例中,它显示了如何在调用close()方法中关闭存储

我也做过类似的事情(我没有在init()方法中启动它们,而是在Scala中使用lazy val),我发现我的Processor close()方法在创建存储之后被多次调用

class EventWindowProcessor(sessionStoreName: String, lastSessionByChannelStoreName: String, lastChannelStoreName: String)
extends AbstractProcesso

// example of a store
private lazy val lastChannelStore: KeyValueStore[MyKey, Channel] =
  context()
    .getStateStore(lastChannelStoreName)
    .asInstanceOf[KeyValueStore[MyKey, Channel]]

override def init(context: ProcessorContext) = {
  super.init(context)
}

override def close() = {
  logger.info("CLOSING PROCESSOR") 
}

override def process(key: String, value: String): Unit = {
    // ... my stuff here
}
因此,我得到了以下输出,显示了processor.close()在拓扑运行开始时被调用了很多次,并且在应用程序的后期也被调用

[2018-06-08 05:13:16,255] INFO Stream Application starting, name: stream-processor (my.package.StreamProcessorApplication$)
[2018-06-08 05:13:16,760] INFO Topology: Sub-topologies:
Sub-topology: 0
Source: event-source (topics: [events])
--> session-processor
Processor: session-processor (stores: [sessionStoreName, lastSessionByChannelStoreName, lastChannelStoreName])
--> error-event-sink, order-sink, pageviews-sink, session-sink
<-- event-source
Sink: error-event-sink (topic: error-events)
<-- session-processor
Sink: order-sink (topic: orders)
<-- session-processor
Sink: pageviews-sink (topic: pageviews)
<-- session-processor
Sink: session-sink (topic: sessions)
<-- session-processor
Global Stores:
none
(my.package.StreamProcessorApplication$)
[2018-06-08 05:14:01,425] INFO CLOSING PROCESSOR (my.package.StreamProcessor)
[2018-06-08 05:14:01,539] INFO CLOSING PROCESSOR (my.package.StreamProcessor)
[2018-06-08 05:14:01,640] INFO CLOSING PROCESSOR (my.package.StreamProcessor)
      ... (102 lines like that)
[2018-06-08 05:29:05,548] INFO   .... my own application logging here 
[2018-06-08 05:13:16255]信息流应用程序启动,名称:流处理器(my.package.StreamProcessorApplication$)
[2018-06-08 05:13:16760]信息拓扑:子拓扑:
子拓扑:0
来源:事件源(主题:[事件])
-->会话处理器
处理器:会话处理器(存储:[sessionStoreName,lastSessionByChannelStoreName,lastChannelStoreName])
-->错误事件接收器、订单接收器、页面视图接收器、会话接收器

文档中的示例不正确。您不应该关闭该存储--该存储由Kafka Streams管理,Kafka Streams将为您的用户关闭该存储。(我将做一个PR来修复代码示例。感谢您指出。)

关于调用
处理器#close()
:处理器可能会关闭并重新打开。这发生在再平衡期间。因此,编写代码的方式必须确保它能够正确地用于多次调用
init()
close()
——我们最近为此更新了JavaDocs(改进的JavaDocs将成为Kafka 2.0版本的一部分)