Apache storm 运行在不同集群上的两个独立风暴拓扑读取同一卡夫卡主题(使用卡夫卡喷口)时出错 公共类WordCountTopology{ 专用静态最终串过滤器\u螺栓\u ID=“过滤器螺栓”; 私有静态最终字符串计数\u螺栓\u ID=“计数螺栓”; 私有静态最终字符串拓扑\u NAME=“字数拓扑”; 静态最终属性配置配置=新属性配置(); 静止的{ 试一试{ CONFIGURATION.load(ClassLoader.getSystemResourceAsStream( “配置/拓扑属性”); }捕获(配置异常e){ 抛出新的RuntimeException(“加载拓扑属性失败”); } } 私有静态KafkaSpout createKafkaSpout(){ List hostList=new ArrayList(); String kafkaPortStr=CONFIGURATION.getString(“KAFKA_端口”); String[]kafkaHosts=CONFIGURATION.getString(“KAFKA_HOSTS”).split(“,”); 用于(字符串卡夫卡霍斯特:卡夫卡霍斯特){ 添加(kafkaHost+“:”+kafkaPortStr); } BrokerHosts hosts=StaticHosts.fromHostString( hostList,CONFIGURATION.getInt(“NUM_KAFKA_分区”); 喷动配置喷动配置=新喷动配置( 主持人,, CONFIGURATION.getString(“卡夫卡主题”), CONFIGURATION.getString(“ZOOKEEPER\u存储路径”), getString(“ZOOKEEPER\u STORAGE\u ID”); 喷口配置强制开始设置时间(-1); 返回新的KafkaSpout(spoutConfig); } topo.properties(非完整文件): 地形1 卡夫卡主题=清漆 ZOOKEEPER\存储\路径=/kafkastrm ZOOKEEPER\u STORAGE\u ID=发现 地形2 卡夫卡主题=清漆 ZOOKEEPER\存储\路径=/kafkastrm8 ZOOKEEPER\u STORAGE\u ID=发现8

Apache storm 运行在不同集群上的两个独立风暴拓扑读取同一卡夫卡主题(使用卡夫卡喷口)时出错 公共类WordCountTopology{ 专用静态最终串过滤器\u螺栓\u ID=“过滤器螺栓”; 私有静态最终字符串计数\u螺栓\u ID=“计数螺栓”; 私有静态最终字符串拓扑\u NAME=“字数拓扑”; 静态最终属性配置配置=新属性配置(); 静止的{ 试一试{ CONFIGURATION.load(ClassLoader.getSystemResourceAsStream( “配置/拓扑属性”); }捕获(配置异常e){ 抛出新的RuntimeException(“加载拓扑属性失败”); } } 私有静态KafkaSpout createKafkaSpout(){ List hostList=new ArrayList(); String kafkaPortStr=CONFIGURATION.getString(“KAFKA_端口”); String[]kafkaHosts=CONFIGURATION.getString(“KAFKA_HOSTS”).split(“,”); 用于(字符串卡夫卡霍斯特:卡夫卡霍斯特){ 添加(kafkaHost+“:”+kafkaPortStr); } BrokerHosts hosts=StaticHosts.fromHostString( hostList,CONFIGURATION.getInt(“NUM_KAFKA_分区”); 喷动配置喷动配置=新喷动配置( 主持人,, CONFIGURATION.getString(“卡夫卡主题”), CONFIGURATION.getString(“ZOOKEEPER\u存储路径”), getString(“ZOOKEEPER\u STORAGE\u ID”); 喷口配置强制开始设置时间(-1); 返回新的KafkaSpout(spoutConfig); } topo.properties(非完整文件): 地形1 卡夫卡主题=清漆 ZOOKEEPER\存储\路径=/kafkastrm ZOOKEEPER\u STORAGE\u ID=发现 地形2 卡夫卡主题=清漆 ZOOKEEPER\存储\路径=/kafkastrm8 ZOOKEEPER\u STORAGE\u ID=发现8,apache-storm,Apache Storm,我正在尝试运行两个tpoplogies,它们使用来自同一个kafka主题的数据,拓扑1可以很好地使用上面类似的代码,但拓扑2给出了错误 我得到卡夫卡喷口以下错误 kafka.common.OffsetAutoFrangeException在 sun.reflect.NativeConstructorAccessorImpl.newInstance0(本机方法)位于 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstr

我正在尝试运行两个tpoplogies,它们使用来自同一个kafka主题的数据,拓扑1可以很好地使用上面类似的代码,但拓扑2给出了错误

我得到卡夫卡喷口以下错误 kafka.common.OffsetAutoFrangeException在 sun.reflect.NativeConstructorAccessorImpl.newInstance0(本机方法)位于
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl

请尝试更改并查看它是否有效
spoutConfig.forceStartOffsetTime(-1);


spoutConfig.forceStartOffsetTime(-2);


这将迫使喷口倒回到以前的偏移量,要了解更多信息,您可以

解决方案是,我必须转到zookeeper并使用zookeeper cli删除拓扑2的zk存储根路径/Kafkastrm8,然后重新提交拓扑,它工作正常

public class WordCountTopology {
   private static final String FILTER_BOLT_ID = "filter-bolt";
   private static final String COUNT_BOLT_ID = "count-bolt";
   private static final String TOPOLOGY_NAME = "word-count-topology";

static final PropertiesConfiguration CONFIGURATION = new PropertiesConfiguration();

static {
try {
  CONFIGURATION.load(ClassLoader.getSystemResourceAsStream(
      "config/topo.properties"));
  } catch (ConfigurationException e) {
  throw new RuntimeException("Failed to load topo properties");
  }
}


private static KafkaSpout createKafkaSpout() {
  List<String> hostList = new ArrayList<String>();
  String kafkaPortStr = CONFIGURATION.getString("KAFKA_PORT");
  String[] kafkaHosts = CONFIGURATION.getString("KAFKA_HOSTS").split(",");
  for (String kafkaHost : kafkaHosts) {
    hostList.add(kafkaHost + ":" + kafkaPortStr);
  }
  BrokerHosts hosts = StaticHosts.fromHostString(
    hostList, CONFIGURATION.getInt("NUM_KAFKA_PARTITIONS"));

  SpoutConfig spoutConfig = new SpoutConfig(
    hosts,
    CONFIGURATION.getString("KAFKA_TOPIC"),
    CONFIGURATION.getString("ZOOKEEPER_STORAGE_PATH"),
    CONFIGURATION.getString("ZOOKEEPER_STORAGE_ID"));
    spoutConfig.forceStartOffsetTime(-1);

  return new KafkaSpout(spoutConfig);
}


topo.properties (not full file):

Topology1
KAFKA_TOPIC=varnish
ZOOKEEPER_STORAGE_PATH=/kafkastorm
ZOOKEEPER_STORAGE_ID=discovery

Topology2
KAFKA_TOPIC=varnish
ZOOKEEPER_STORAGE_PATH=/kafkastorm8
ZOOKEEPER_STORAGE_ID=discovery8