Apache kafka 卡夫卡使用者路径不能以/character结尾

Apache kafka 卡夫卡使用者路径不能以/character结尾,apache-kafka,messagebroker,kafka-consumer-api,Apache Kafka,Messagebroker,Kafka Consumer Api,我正在使用ApacheKafka 0.8.2.1将web事件流式传输到其他数据源。我写的卡夫卡制作人工作得很好,当我运行Kafka-console-consumer.sh时,我能够看到数据流通过我的主题。然而,我没有任何运气试图让我的卡夫卡消费者检索这些消息。有什么想法吗 当我的代码试图运行consumer.createMessageStreams(topicCountMap)时,输出了以下关于不正确路径的错误 这是我的卡夫卡消费者的代码 val consumer: ConsumerConn

我正在使用ApacheKafka 0.8.2.1将web事件流式传输到其他数据源。我写的卡夫卡制作人工作得很好,当我运行Kafka-console-consumer.sh时,我能够看到数据流通过我的主题。然而,我没有任何运气试图让我的卡夫卡消费者检索这些消息。有什么想法吗

当我的代码试图运行consumer.createMessageStreams(topicCountMap)时,输出了以下关于不正确路径的错误

这是我的卡夫卡消费者的代码

  val consumer: ConsumerConnector = kafka.consumer.Consumer.createJavaConsumerConnector(createConsumerConfig())

  var executor: ExecutorService = null

  def run(a_numThreads: Integer) {
    var topicCountMap: java.util.Map[String, Integer] = new java.util.HashMap[String, Integer]()

    topicCountMap.put("testEvent", new Integer(a_numThreads))

    var consumerMap = consumer.createMessageStreams(topicCountMap)

    var streams = consumerMap.get("testEvent")
    // now launch all the threads
    executor = Executors.newFixedThreadPool(a_numThreads)

    // now create an object to consume the messages
    //
    var threadNumber: Integer = 0
    var streamsItr = streams.iterator()
    while (streamsItr.hasNext()) {
      var stream = streamsItr.next()
      executor.submit(new EventConsumer(stream, threadNumber))
      threadNumber = threadNumber + 1
    }
  }

  def createConsumerConfig(): ConsumerConfig = {
    var props: Properties = new Properties()
    props.put("zookeeper.connect", "127.0.0.1:2181")
    props.put("zk.connect", "127.0.0.1:2181")
    props.put("group.id", "testConsumer")
    props.put("groupid", "tesConsumer")
    props.put("zookeeper.session.timeout.ms", "400")
    props.put("zookeeper.sync.time.ms", "200")
    props.put("auto.commit.interval.ms", "1000")

    return new ConsumerConfig(props)
  }

Spark CheckpointWriter在无法访问存储的检查点路径时生成此异常消息。请确保已禁用检查点或提供正确的路径。在连接成功后发生异常

位于org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:58)

似乎写入程序无法访问将保存检查点信息的目录


这可能不是实际问题,但为什么您同时需要
zookeeper.connect
zk.connect
?另外,
group.id
&
groupid
?在某个时候,卡夫卡团队更改了消费者使用的配置属性,不一定正确更新了文档。不幸的是,我不知道它使用的是哪一个,所以我将它们都包括在内。从0.8开始,您需要zookeeper.connect&group.id。。与异常相关,这看起来与zookeeper有关,但您的配置对我来说似乎没问题。。如果可能的话,你可以深入了解更多细节。。希望能给我们一些线索
  val consumer: ConsumerConnector = kafka.consumer.Consumer.createJavaConsumerConnector(createConsumerConfig())

  var executor: ExecutorService = null

  def run(a_numThreads: Integer) {
    var topicCountMap: java.util.Map[String, Integer] = new java.util.HashMap[String, Integer]()

    topicCountMap.put("testEvent", new Integer(a_numThreads))

    var consumerMap = consumer.createMessageStreams(topicCountMap)

    var streams = consumerMap.get("testEvent")
    // now launch all the threads
    executor = Executors.newFixedThreadPool(a_numThreads)

    // now create an object to consume the messages
    //
    var threadNumber: Integer = 0
    var streamsItr = streams.iterator()
    while (streamsItr.hasNext()) {
      var stream = streamsItr.next()
      executor.submit(new EventConsumer(stream, threadNumber))
      threadNumber = threadNumber + 1
    }
  }

  def createConsumerConfig(): ConsumerConfig = {
    var props: Properties = new Properties()
    props.put("zookeeper.connect", "127.0.0.1:2181")
    props.put("zk.connect", "127.0.0.1:2181")
    props.put("group.id", "testConsumer")
    props.put("groupid", "tesConsumer")
    props.put("zookeeper.session.timeout.ms", "400")
    props.put("zookeeper.sync.time.ms", "200")
    props.put("auto.commit.interval.ms", "1000")

    return new ConsumerConfig(props)
  }