Java 从KafkaSpout读取并将其写回Kafka Bolt的示例

我正在尝试编写一个基于Storm的代码，从一个主题读取消息，然后写回另一个主题。输入主题具有ProtoBuf格式的数据，输出将具有JSON格式。我无法做到这一点

以下是构建拓扑的代码：

    Config conf = new Config();
    //set producer properties.
    Properties props = new Properties();
    props.put("bootstrap.servers", "localhost:9093");
    props.put("request.required.acks", "1");
    props.put("key.serializer", "org.apache.kafka.common.serialization.ByteArraySerializer");
    props.put("value.serializer", "org.apache.kafka.common.serialization.ByteArraySerializer");
    conf.put("kafka.broker.config", props);
    conf.put(KafkaBolt.TOPIC, "out-storm");
    KafkaBolt bolt = new KafkaBolt()
            .withProducerProperties(props)
            .withTopicSelector(new DefaultTopicSelector("out-storm")).withTupleToKafkaMapper(new FieldNameBasedTupleToKafkaMapper<String, String>());        

    BrokerHosts hosts = new ZkHosts("localhost:2181");
    SpoutConfig spoutConfig = new SpoutConfig(hosts, "incoming-server", "/" + "incoming-server",
            UUID.randomUUID().toString());
    spoutConfig.scheme = new SchemeAsMultiScheme(new StringScheme());
    KafkaSpout kafkaSpout = new KafkaSpout(spoutConfig);
    TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout("kafka-spout", kafkaSpout);
    builder.setBolt("lookup-bolt", new ReportBolt(),4).shuffleGrouping("kafka-spout");
    builder.setBolt("kafka-producer-spout", bolt).shuffleGrouping("lookup-bolt");
    LocalCluster cluster = new LocalCluster();
    Config config = new Config();
    config.setDebug(true);
    config.put(Config.TOPOLOGY_MAX_SPOUT_PENDING, 1);
    config.put("kafka.broker.config", props);
    config.put(KafkaBolt.TOPIC, "out-storm");
    cluster.submitTopology("KafkaStormSample", config, builder.createTopology());
    Thread.sleep(1000000);

---

我认为问题在于您在bootstrap.server配置中引用了错误的端口。试着换成9092

    System.out.println("HELLO " + input);
    JSONObject jo= new JSONObject(); 
    for (String f:input.getFields()){
        jo.put(f, input.getValueByField(f));
    }
    collector.ack(input);
    List<Object> list = new ArrayList<Object>();
    list.add(jo);
    collector.emit(list);

      5207 [main] WARN  o.a.s.d.nimbus - Topology submission exception. (topology name='KafkaStormSample') #error {
 :cause nil
 :via
 [{:type org.apache.storm.generated.InvalidTopologyException
   :message nil
   :at [org.apache.storm.daemon.common$validate_structure_BANG_ invoke common.clj 181]}]
 :trace
 [[org.apache.storm.daemon.common$validate_structure_BANG_ invoke common.clj 181]
  [org.apache.storm.daemon.common$system_topology_BANG_ invoke common.clj 360]
  [org.apache.storm.daemon.nimbus$fn__7064$exec_fn__2461__auto__$reify__7093 submitTopologyWithOpts nimbus.clj 1512]
  [org.apache.storm.daemon.nimbus$fn__7064$exec_fn__2461__auto__$reify__7093 submitTopology nimbus.clj 1544]
  [sun.reflect.NativeMethodAccessorImpl invoke0 NativeMethodAccessorImpl.java -2]
  [sun.reflect.NativeMethodAccessorImpl invoke NativeMethodAccessorImpl.java 62]
  [sun.reflect.DelegatingMethodAccessorImpl invoke DelegatingMethodAccessorImpl.java 43]
  [java.lang.reflect.Method invoke Method.java 497]
  [clojure.lang.Reflector invokeMatchingMethod Reflector.java 93]
  [clojure.lang.Reflector invokeInstanceMethod Reflector.java 28]
  [org.apache.storm.testing$submit_local_topology invoke testing.clj 301]
  [org.apache.storm.LocalCluster$_submitTopology invoke LocalCluster.clj 49]
  [org.apache.storm.LocalCluster submitTopology nil -1]
  [com.mediaiq.StartStorm main StartStorm.java 81]]}