Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/java/326.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/8/linq/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java 部署Storm build JAR_Java_Apache Storm_Kafka Consumer Api - Fatal编程技术网

Java 部署Storm build JAR

Java 部署Storm build JAR,java,apache-storm,kafka-consumer-api,Java,Apache Storm,Kafka Consumer Api,我开发了一个Java类,它从Kafka队列读取数据并打印出来 ZkHosts zkHosts=new ZkHosts("localhost:2181"); String topic_name="test"; String consumer_group_id="storm"; String zookeeper_root=""; SpoutConfig kafkaConfig=new SpoutConfig(zkHosts, topic_name, zookeeper_root,

我开发了一个Java类,它从Kafka队列读取数据并打印出来

ZkHosts zkHosts=new ZkHosts("localhost:2181");
String topic_name="test";
String consumer_group_id="storm";
String zookeeper_root="";
SpoutConfig kafkaConfig=new SpoutConfig(zkHosts, 
        topic_name, zookeeper_root, consumer_group_id);
kafkaConfig.scheme=new SchemeAsMultiScheme(new StringScheme());
/*kafkaConfig.forceFromStart=false;
kafkaConfig.startOffsetTime =-2;*/

KafkaSpout kafkaSpout = new KafkaSpout(kafkaConfig);
TopologyBuilder builder=new TopologyBuilder();
//builder.setSpout("KafkaSpout", kafkaSpout, 1);
builder.setSpout("KafkaSpout", kafkaSpout);
builder.setBolt("PrinterBolt", new PrinterBolt()).globalGrouping("KafkaSpout");
Map<String, Object> conf = new HashMap<String, Object>();
conf.put(Config.TRANSACTIONAL_ZOOKEEPER_PORT, 2181);
conf.put(Config.TRANSACTIONAL_ZOOKEEPER_SERVERS, Arrays.asList("localhost"));
conf.put(Config.STORM_ZOOKEEPER_SESSION_TIMEOUT, 20000);
conf.put(Config.STORM_ZOOKEEPER_CONNECTION_TIMEOUT, 20000);
conf.put(Config.STORM_ZOOKEEPER_RETRY_TIMES, 3);
conf.put(Config.STORM_ZOOKEEPER_RETRY_INTERVAL, 30);
LocalCluster cluster=new LocalCluster();
try{
    cluster.submitTopology("KafkaConsumerTopology", conf, builder.createTopology());
    Thread.sleep(120000);
}catch (Exception e) {
    //throw new IllegalStateException("Couldn't initialize the topology", e);
    System.out.println(e.getMessage());
}

@Matthias J.Sax和所有人,谢谢你们的帮助。 我在这里犯的错误是,我遵循的部署过程是错误的。 要部署Topology构建,我必须遵循以下流程:

  • Jar必须被推到storm AWS文件夹中,然后必须运行下面的命令,以使其被storm识别
  • rm-f*.out

    (诺胡普垃圾箱/风暴雨云>尼穆布斯。外)&

    (nohup垃圾箱/风暴监督员>监督员外出)&

    (nohup bin/storm jar topos/IoT.jar com.bridgera.IoT.test.App01>IoT.out)&

    在这里,我告诉storm它可以在哪里找到我的jar和它可以在哪里找到拓扑生成器的主类


    谢谢大家…

    错误的堆栈跟踪会很有帮助。@MatthiasJ.Sax,我在上面添加了。请检查一下。@MatthiasJ.Sax,你对这一行有什么想法吗,kafka.consumer.SimpleConsumer.getOffsetsBefore(未知源)~[uber-iot-0.0.1-SNAPSHOT.jar:na]kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(未知源)~[uber-iot-0.0.1-SNAPSHOT.jar:na]@MatthiasJ.Sax,如何将offsetBefore参数提供给简单的使用者?似乎您提供了两个日志库:
    log4j-over-slf4j.jar和slf4j-log4j12.jar
    ——Storm 0.9.x正在使用
    log4j-over-slf4j.jar
    ——尝试删除
    slf4j-log4j12.jar
    29537 [Thread-14-KafkaSpout] ERROR backtype.storm.util - Async loop died!
    java.lang.ExceptionInInitializerError: null
        at org.apache.log4j.Logger.getLogger(Logger.java:39) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at kafka.utils.Logging$class.logger(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at kafka.network.BlockingChannel.logger$lzycompute(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at kafka.network.BlockingChannel.logger(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at kafka.utils.Logging$class.debug(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at kafka.network.BlockingChannel.debug(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at kafka.network.BlockingChannel.connect(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at kafka.consumer.SimpleConsumer.connect(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at kafka.consumer.SimpleConsumer.getOrMakeConnection(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at kafka.consumer.SimpleConsumer.getOffsetsBefore(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(Unknown Source) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:77) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:67) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at storm.kafka.PartitionManager.<init>(PartitionManager.java:83) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:98) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:135) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at backtype.storm.daemon.executor$fn__3373$fn__3388$fn__3417.invoke(executor.clj:565) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        at clojure.lang.AFn.run(AFn.java:24) [uber-iot-0.0.1-SNAPSHOT.jar:na]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
    Caused by: java.lang.IllegalStateException: Detected both log4j-over-slf4j.jar AND slf4j-log4j12.jar on the class path, preempting StackOverflowError. See also http://www.slf4j.org/codes.html#log4jDelegationLoop for more details.
        at org.apache.log4j.Log4jLoggerFactory.<clinit>(Log4jLoggerFactory.java:49) ~[uber-iot-0.0.1-SNAPSHOT.jar:na]
        ... 22 common frames omitted