Node.js can'；t连接到本地网络中的kafka容器

我在ubuntu 18.04中运行docker yaml文件中的zookeeper和kafka实例

version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka
    ports:
      - "9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_CREATE_TOPICS: "test-topic:5:2"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

它正在发挥作用

CONTAINER ID   IMAGE                       COMMAND                  CREATED         STATUS         PORTS                                                NAMES
d3d84a6b39f7   wurstmeister/kafka          "start-kafka.sh"         3 minutes ago   Up 3 minutes   0.0.0.0:49157->9092/tcp                              desktop_kafka_1
b2012f08b3f9   wurstmeister/zookeeper      "/bin/sh -c '/usr/sb…"   5 hours ago     Up 3 minutes   22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp   desktop_zookeeper_1

但是，卡夫卡客户端未能将卡夫卡连接为

const { Kafka,logLevel,CompressionCodecs,CompressionTypes } = require('kafkajs');
const kafka = new Kafka({
  logLevel: logLevel.DEBUG,
  brokers: ['localhost:9092'],   // tried on ['192.168.1.6:9092']
  clientId: 'example-producer',
})

const topic = 'topic-test'
const producer = kafka.producer()

const getRandomNumber = () => Math.round(Math.random(10) * 1000)
const createMessage = num => ({
  key: `key-${num}`,
  value: `value-${num}-${new Date().toISOString()}`,
})

const sendMessage = () => {
  return producer
    .send({
      topic,
      compression: CompressionTypes.GZIP,
      messages: Array(getRandomNumber())
        .fill()
        .map(_ => createMessage(getRandomNumber())),
    })
    .then(console.log)
    .catch(e => console.error(`[example/producer] ${e.message}`, e))
}

const run = async () => {
  await producer.connect()
  setInterval(sendMessage, 3000)
}

run().catch(e => console.error(`[example/producer] ${e.message}`, e))

代码输出

    [example/producer] Connection error: connect ECONNREFUSED 127.0.0.1:9092 KafkaJSNonRetriableError
  Caused by: KafkaJSConnectionError: Connection error: connect ECONNREFUSED 127.0.0.1:9092
    at Socket.onError (/home/xsz/Desktop/node_modules/kafkajs/src/network/connection.js:152:23)
    at Socket.emit (events.js:314:20)
    at emitErrorNT (internal/streams/destroy.js:92:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {
  name: 'KafkaJSNumberOfRetriesExceeded',
  retriable: false,
  helpUrl: undefined,
  originalError: KafkaJSConnectionError: Connection error: connect ECONNREFUSED 127.0.0.1:9092
      at Socket.onError (/home/xsz/Desktop/node_modules/kafkajs/src/network/connection.js:152:23)
      at Socket.emit (events.js:314:20)
      at emitErrorNT (internal/streams/destroy.js:92:8)
      at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
      at processTicksAndRejections (internal/process/task_queues.js:84:21) {
    retriable: true,
    helpUrl: undefined,
    broker: 'localhost:9092',
    code: 'ECONNREFUSED'
  },
  retryCount: 5,
  retryTime: 10304
}

在docker-compose.ymal中，KAFKA_播发的_主机_名称被配置为localhost或192.168.1.6（主机的本地ip地址），两者都显示与上述相同的错误

注: 使用

ip地址显示命令输出

并在主机上运行代码

const kafka = new Kafka({
  logLevel: logLevel.DEBUG,
  brokers: ['172.17.0.1:9092'],   // tried on ['192.168.1.6:9092']
  clientId: 'example-producer',
})

---

但是仍然面临着同样的问题，有什么不对呢

需要做出一些改变

首先，将连接主机更改为

localhost

const kafka = new Kafka({
    logLevel: logLevel.DEBUG,
    brokers: ['localhost:9092'],  
    clientId: 'example-producer',
})

然后在docker内部编写文件 更改

端口

并添加

链接

version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka
    links:
      - zookeeper
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_CREATE_TOPICS: "test-topic:5:2"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

最后，在编写文件时只使用一个代理

我使用Kafka-tool2.0查看您的配置，似乎只有一个分区用于“主题测试”，这就是为什么您会看到此错误，请使用更多代理来避免此错误

 {"level":"DEBUG","timestamp":"2020-12-15T07:54:05.315Z","logger":"kafkajs","message":"[Connection] Request Metadata(key: 3, version: 6)","broker":"localhost:9092","clientId":"example-producer","correlationId":25,"expectResponse":true,"size":47}There is no listener on the leader broker that matches the listener on which metadata request was processed KafkaJSNonRetriableError Caused by: KafkaJSProtocolError: There is no listener on the leader broker that matches the listener on which metadata request was processed.

在LinuxMint20上品尝

输出：

{"level":"DEBUG","timestamp":"2020-12-15T06:55:59.662Z","logger":"kafkajs","message":"[Connection] Response Produce(key: 0, version: 7)","broker":"localhost:9092","clientId":"example-producer","correlationId":46,"size":58,"data":{"topics":[{"topicName":"topic-test","partitions":[{"partition":0,"errorCode":0,"baseOffset":"18379","logAppendTime":"-1","logStartOffset":"0"}]}],"throttleTime":0,"clientSideThrottleTime":0}}
[
  {
    topicName: 'topic-test',
    partition: 0,
    errorCode: 0,
    baseOffset: '18379',
    logAppendTime: '-1',
    logStartOffset: '0'
  }
]

---

您尚未在docker compose文件中正确映射kafka实例的端口，您应该进行更改

ports:
  - "9092"

与

现在，您可以为容器分配任意端口

---

0.0.0.0:49157->9092/tcp

卡夫卡似乎已连接，但另一个问题是{“级别”：“调试”，“时间戳”：“2020-12-15T07:54:05.315Z”，“记录器”：“卡夫卡”，“消息”：“[连接]请求元数据（键：3，版本：6）”，“代理”：“本地主机：9092”，“客户端ID”：“示例生产者”，“相关ID”：25，“预期响应”：true，“大小”：47}leader broker上没有与处理元数据请求的侦听器匹配的侦听器KafkajsNonRetrieableError导致的错误：KafkaJSProtocolError:leader broker上没有与处理元数据请求的侦听器匹配的侦听器。我使用Kafka-tool2.0查看您的配置，似乎没有打开只有一个分区用于“主题测试”这就是你看到这个错误的方式我明白了，你是说测试主题：5:2，它是5个分区和2个replia。如果我需要创建5个分区，那么我需要用5个kafka实例构建一个kafka集群，不是吗

{"level":"DEBUG","timestamp":"2020-12-15T06:55:59.662Z","logger":"kafkajs","message":"[Connection] Response Produce(key: 0, version: 7)","broker":"localhost:9092","clientId":"example-producer","correlationId":46,"size":58,"data":{"topics":[{"topicName":"topic-test","partitions":[{"partition":0,"errorCode":0,"baseOffset":"18379","logAppendTime":"-1","logStartOffset":"0"}]}],"throttleTime":0,"clientSideThrottleTime":0}}
[
  {
    topicName: 'topic-test',
    partition: 0,
    errorCode: 0,
    baseOffset: '18379',
    logAppendTime: '-1',
    logStartOffset: '0'
  }
]

ports:
  - "9092"

ports:
  - "9092:9092"