如何确保docker kafka服务中的唯一代理ID

我想创建一个可伸缩的kafka集群，但每个kafka实例都需要一个唯一的代理ID。有什么好方法可以确保这一点

下面是我用来部署应用程序的compose文件。我正在使用

docker stack deploy--compose文件docker-compose.yml mystack

将其部署到我的docker swarm。目前，代理ID设置为1，但当我将kafka实例的数量从一个增加到两个（例如，

docker service scale=2

）时，这将不起作用，而这正是我试图解决的问题

---
version: '3'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_CLIENT_PORT: 22181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 5
      ZOOKEEPER_SYNC_LIMIT: 2
      ZOOKEEPER_SERVERS: localhost:22888:23888
    network_mode: host
    extra_hosts:
      - "moby:127.0.0.1"
  kafka:
    image: confluentinc/cp-kafka:latest
    network_mode: host
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:22181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:19092
    extra_hosts:
      - "moby:127.0.0.1"

---

我很乐意回答你的问题。。您是否尝试使用环境变量。。。像这样

version: '3'
services:
     zookeeper:
        image: confluentinc/cp-zookeeper:latest
        environment:
          ZOOKEEPER_SERVER_ID: 1
          ZOOKEEPER_CLIENT_PORT: 22181
          ZOOKEEPER_TICK_TIME: 2000
          ZOOKEEPER_INIT_LIMIT: 5
          ZOOKEEPER_SYNC_LIMIT: 2
          ZOOKEEPER_SERVERS: localhost:22888:23888
        network_mode: host
        extra_hosts:
          - "moby:127.0.0.1"
      kafka:
        image: confluentinc/cp-kafka:latest
        network_mode: host
        depends_on:
          - zookeeper
        environment:
          KAFKA_BROKER_ID: ${TAG}
          KAFKA_ZOOKEEPER_CONNECT: zookeeper:22181
          KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:19092
        extra_hosts:
          - "moby:127.0.0.1"

请注意对KAFKA_BROKER_ID所做的替换，如果服务在同一个节点上进行了扩展，您也可能会遇到端口问题，但这是一个需要解决的不同问题

---
version: '3'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_CLIENT_PORT: 22181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 5
      ZOOKEEPER_SYNC_LIMIT: 2
      ZOOKEEPER_SERVERS: localhost:22888:23888
    network_mode: host
    extra_hosts:
      - "moby:127.0.0.1"
  kafka:
    image: confluentinc/cp-kafka:latest
    network_mode: host
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:22181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:19092
    extra_hosts:
      - "moby:127.0.0.1"

因此，在放大之前，只需将变量的值更改为不同的值，您可能还需要编写脚本

更多信息可以在这里找到

服务启动后，您可以在控制台日志中看到代理id

kafka_1      | KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:19092
zookeeper_1  | ZOOKEEPER_SYNC_LIMIT=2
kafka_1      | KAFKA_BROKER_ID=9
zookeeper_1  | ZOOKEEPER_TICK_TIME=2000
kafka_1      | KAFKA_VERSION=1.0.0
kafka_1      | KAFKA_ZOOKEEPER_CONNECT=zookeeper:22181
zookeeper_1  | ZULU_OPENJDK_VERSION=8=8.17.0.3
kafka_1      | LANG=C.UTF-8
zookeeper_1  | _=/usr/bin/env

---

事实证明，默认情况下，如果您不为kafka分配代理ID，zookeeper将自动为kafka分配唯一的代理ID。因此，您可以安全地向上或向下扩展kafka服务，而无需考虑代理ID的管理。有关更多信息，请参阅（搜索表中相关行的“zookeeper序列从MaxReservedBrokerId+1开始”）。默认情况下，MaxReservedBrokerId为1000，因此您的第一个自动分配的代理ID将为1001，并且当您添加其他kafka容器时，zookeeper将按顺序分配。因此，您的第二个kafka容器将具有代理ID 1002，等等