Java 卡夫卡风暴的定义

为了制作storm kafka 3分区，我有以下代码

public class Topology {
    private static final String topic1 = "Real1";
    private static final String topic2 = "Real2";

    public static void main(String[] args) throws AlreadyAliveException,
        InvalidTopologyException, IOException {

        BasicConfigurator.configure();

        Broker brokerForPartition0 = new Broker("localhost:9092");
        Broker brokerForPartition1 = new Broker("localhost:9092");
        Broker brokerForPartition2 = new Broker("localhost:9092");
        GlobalPartitionInformation partitionInfo = new GlobalPartitionInformation();
        partitionInfo.addPartition(0, brokerForPartition0);
        partitionInfo.addPartition(1, brokerForPartition1);
        partitionInfo.addPartition(2, brokerForPartition2);
        StaticHosts hosts = new StaticHosts(partitionInfo);

        SpoutConfig kafkaConfig1 = new SpoutConfig(hosts, topic1,
            "/kafkaStorm", "Real1KafkaSpout");

        SpoutConfig kafkaConfig2 = new SpoutConfig(hosts, topic2,
            "/kafkaStorm", "Real2KafkaSpout");

        kafkaConfig1.zkServers = ImmutableList.of("localhost");
        kafkaConfig1.zkPort = 2181;

        kafkaConfig1.scheme = new SchemeAsMultiScheme(new StringScheme());
        kafkaConfig1.forceFromStart = true;
        kafkaConfig1.startOffsetTime = kafka.api.OffsetRequest.EarliestTime();

        kafkaConfig2.zkServers = ImmutableList.of("localhost");
        kafkaConfig2.zkPort = 2181;
        kafkaConfig2.scheme = new SchemeAsMultiScheme(new StringScheme());
        kafkaConfig2.forceFromStart = true;
        kafkaConfig2.startOffsetTime = kafka.api.OffsetRequest.EarliestTime();

        TopologyBuilder builder = new TopologyBuilder();

        builder.setSpout("Real1", new KafkaSpout(kafkaConfig1), 2);
        builder.setSpout("Real2", new KafkaSpout(kafkaConfig2), 2);

        builder.setBolt("StreamMerging", new StreamMergingBolt(), 2)
            .setNumTasks(2).shuffleGrouping("Real1")
            .shuffleGrouping("Real2");

        Config config = new Config();

        config.setDebug(false);
        config.setMaxSpoutPending(10000);

        if (args.length == 0) {
            LocalCluster cluster = new LocalCluster();

            cluster.submitTopology("Topology", config, builder.createTopology());
            cluster.killTopology("Topology");
            cluster.shutdown();
        } else {
            StormSubmitter.submitTopology(args[0], config,
                builder.createTopology());
        }

        try {
            Thread.sleep(6000);
        } catch (InterruptedException ex) {
            ex.printStackTrace();
        }
    }
}

我得到以下错误，任何人都可以帮助我。要使此拓扑工作，每个主题有3个分区。我正在使用2

KafkaSpout

s，不同的主题出现在同一个螺栓上

java.nio.channels.ClosedChannelException at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
  at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:78)
  at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$sendRequest(SimpleConsumer.scala:68)
  at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:127)
  at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:79)
  at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:77)
  at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:67)
  at storm.kafka.PartitionManager.<init>(PartitionManager.java:83)
  at storm.kafka.StaticCoordinator.<init>(StaticCoordinator.java:34)
  at storm.kafka.KafkaSpout.open(KafkaSpout.java:92)
  at backtype.storm.daemon.executor$fn__3839$fn__3854.invoke(executor.clj:542)
  at backtype.storm.util$async_loop$fn__451.invoke(util.clj:463)
  at clojure.lang.AFn.run(AFn.java:24)
  at java.lang.Thread.run(Thread.java:745) 

kafka.network.BlockingChannel.send处的java.nio.channels.ClosedChannelException（BlockingChannel.scala:100） 在kafka.consumer.SimpleConsumer.liftedTree1$1（SimpleConsumer.scala:78） 位于kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$sendRequest（SimpleConsumer.scala:68） 在kafka.consumer.SimpleConsumer.getOffsetsBefore（SimpleConsumer.scala:127）上 在kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore（SimpleConsumer.scala:79）上 atstorm.kafka.KafkaUtils.getOffset（KafkaUtils.java:77） atstorm.kafka.KafkaUtils.getOffset（KafkaUtils.java:67） PartitionManager.（PartitionManager.java:83） StaticCoordinator.（StaticCoordinator.java:34） atstorm.kafka.KafkaSpout.open（KafkaSpout.java:92） 在backtype.storm.daemon.executor$fn\uuuu3839$fn\uuuu3854.invoke（executor.clj:542） 在backtype.storm.util$async\u循环$fn\uu 451.invoke处（util.clj:463） 在clojure.lang.AFn.run（AFn.java:24） 运行（Thread.java:745）

---

@Matthias J.Sax嗨，我能得到一些建议吗？我对卡夫卡没有太多经验。。。这两个主题（

Real1

和

Real2

都有3个分区吗？也许你不能共享同一个

Broker

，

GlobalPartitionInformation

，或者

StaticHosts

实例，用于这两个

KafkaSpout

的

3个分区（只是一个想法）？，但即使对于一个主题3分区，我也有一个代理，我也会遇到同样的错误