在kubernetes集群外访问bitnami/kafka

我目前正在使用bitnami/kafka image（），并将其部署在kubernetes上

• 库伯内特斯大师：1
• kubernetes工人：3名

在集群中，其他应用程序可以找到kafka。尝试从集群外部访问kafka容器时会出现问题。在阅读《点点滴滴》时，我了解到我们需要为外部kafka客户端设置属性“advised.listener=PLAINTTEXT://hostname:port_number”

我目前正在引用“”。在我添加的values.yaml文件中

值。yaml

    - name: KAFKA_CFG_ADVERTISED_LISTENERS
      value: 'PLAINTEXT://{{ .Values.advertisedListeners }}:9092' 

    - name: KAFKA_CFG_ADVERTISED_LISTENERS
      {{- if $MY_POD_NAME := "kafka-0" }}
      value: 'PLAINTEXT://{{ .Values.advertisedListeners1 }}:9092'
      {{- else if $MY_POD_NAME := "kafka-1" }}
      value: 'PLAINTEXT://{{ .Values.advertisedListeners2 }}:9092'
      {{- else if $MY_POD_NAME := "kafka-2" }}
      value: 'PLAINTEXT://{{ .Values.advertisedListeners3 }}:9092'
      {{- end }}

• 广告列表1:10.21.0.191

和statefulset.yaml

    - name: KAFKA_CFG_ADVERTISED_LISTENERS
      value: 'PLAINTEXT://{{ .Values.advertisedListeners }}:9092' 

    - name: KAFKA_CFG_ADVERTISED_LISTENERS
      {{- if $MY_POD_NAME := "kafka-0" }}
      value: 'PLAINTEXT://{{ .Values.advertisedListeners1 }}:9092'
      {{- else if $MY_POD_NAME := "kafka-1" }}
      value: 'PLAINTEXT://{{ .Values.advertisedListeners2 }}:9092'
      {{- else if $MY_POD_NAME := "kafka-2" }}
      value: 'PLAINTEXT://{{ .Values.advertisedListeners3 }}:9092'
      {{- end }}

对于单个卡夫卡实例，它工作正常。

但对于3节点kafka群集，我更改了一些配置，如下所示： 值。yaml

    - name: KAFKA_CFG_ADVERTISED_LISTENERS
      value: 'PLAINTEXT://{{ .Values.advertisedListeners }}:9092' 

    - name: KAFKA_CFG_ADVERTISED_LISTENERS
      {{- if $MY_POD_NAME := "kafka-0" }}
      value: 'PLAINTEXT://{{ .Values.advertisedListeners1 }}:9092'
      {{- else if $MY_POD_NAME := "kafka-1" }}
      value: 'PLAINTEXT://{{ .Values.advertisedListeners2 }}:9092'
      {{- else if $MY_POD_NAME := "kafka-2" }}
      value: 'PLAINTEXT://{{ .Values.advertisedListeners3 }}:9092'
      {{- end }}

• 广告列表1:10.21.0.191
• 广告商2:10.21.0.192
• 广告商3:10.21.0.193

和Statefulset.yaml

    - name: KAFKA_CFG_ADVERTISED_LISTENERS
      value: 'PLAINTEXT://{{ .Values.advertisedListeners }}:9092' 

    - name: KAFKA_CFG_ADVERTISED_LISTENERS
      {{- if $MY_POD_NAME := "kafka-0" }}
      value: 'PLAINTEXT://{{ .Values.advertisedListeners1 }}:9092'
      {{- else if $MY_POD_NAME := "kafka-1" }}
      value: 'PLAINTEXT://{{ .Values.advertisedListeners2 }}:9092'
      {{- else if $MY_POD_NAME := "kafka-2" }}
      value: 'PLAINTEXT://{{ .Values.advertisedListeners3 }}:9092'
      {{- end }}

预期的结果是，所有3个kafka实例都应将Adverted.listener属性设置为worker nodes ip address

例如：

• 卡夫卡-0-->“明文：//10.21.0.191:9092”

• 卡夫卡-1-->“明文：//10.21.0.192:9092”

• 卡夫卡-3-->“明文：//10.21.0.193:9092”

目前只有一个卡夫卡吊舱处于启动和运行状态，另外两个将进入崩溃状态

另外两个吊舱显示错误为：

[2019-10-20 13:09:37753]信息[LogDirFailureHandler]：启动（kafka.server.ReplicaManager$LogDirFailureHandler） [2019-10-20 13:09:37786]错误[KafkaServer id=1002]在KafkaServer启动期间发生致命错误。准备关闭（kafka.server.KafkaServer） java.lang.IllegalArgumentException:需求失败：播发侦听器中配置的端点10.21.0.191:9092已由代理1001注册 在scala.Predef$.require处（Predef.scala:224） 位于kafka.server.KafkaServer$$anonfun$createBrokerInfo$2.apply（KafkaServer.scala:399） 位于kafka.server.KafkaServer$$anonfun$createBrokerInfo$2.apply（KafkaServer.scala:397） 位于scala.collection.mutable.resizeblearray$class.foreach（resizeblearray.scala:59） 位于scala.collection.mutable.ArrayBuffer.foreach（ArrayBuffer.scala:48） 位于kafka.server.KafkaServer.createBrokerInfo（KafkaServer.scala:397） 在kafka.server.KafkaServer.startup（KafkaServer.scala:261） 在kafka.server.KafkaServerStartable.startup（KafkaServerStartable.scala:38） 在kafka.kafka$.main（kafka.scala:84） 在卡夫卡.卡夫卡.梅因（卡夫卡.斯卡拉）

这意味着statefulset.yaml中应用的逻辑不起作用。 有人能帮我解决这个问题吗

任何帮助都将不胜感激

kubectl get statefulset kafka-o yaml的输出

apiVersion: apps/v1
kind: StatefulSet
metadata:
  creationTimestamp: "2019-10-29T07:04:12Z"
  generation: 1
  labels:
    app.kubernetes.io/component: kafka
    app.kubernetes.io/instance: kafka
    app.kubernetes.io/managed-by: Tiller
    app.kubernetes.io/name: kafka
    helm.sh/chart: kafka-6.0.1
  name: kafka
  namespace: default
  resourceVersion: "12189730"
  selfLink: /apis/apps/v1/namespaces/default/statefulsets/kafka
  uid: d40cfd5f-46a6-49d0-a9d3-e3a851356063
spec:
  podManagementPolicy: Parallel
  replicas: 3
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: kafka
      app.kubernetes.io/instance: kafka
      app.kubernetes.io/name: kafka
  serviceName: kafka-headless
  template:
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/component: kafka
        app.kubernetes.io/instance: kafka
        app.kubernetes.io/managed-by: Tiller
        app.kubernetes.io/name: kafka
        helm.sh/chart: kafka-6.0.1
      name: kafka
    spec:
      containers:
      - env:
        - name: MY_POD_IP
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: status.podIP
        - name: MY_POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: KAFKA_CFG_ZOOKEEPER_CONNECT
          value: kafka-zookeeper
        - name: KAFKA_PORT_NUMBER
          value: "9092"
        - name: KAFKA_CFG_LISTENERS
          value: PLAINTEXT://:$(KAFKA_PORT_NUMBER)
        - name: KAFKA_CFG_ADVERTISED_LISTENERS
          value: PLAINTEXT://10.21.0.191:9092
        - name: ALLOW_PLAINTEXT_LISTENER
          value: "yes"
        - name: KAFKA_CFG_BROKER_ID
          value: "-1"
        - name: KAFKA_CFG_DELETE_TOPIC_ENABLE
          value: "false"
        - name: KAFKA_HEAP_OPTS
          value: -Xmx1024m -Xms1024m
        - name: KAFKA_CFG_LOG_FLUSH_INTERVAL_MESSAGES
          value: "10000"
        - name: KAFKA_CFG_LOG_FLUSH_INTERVAL_MS
          value: "1000"
        - name: KAFKA_CFG_LOG_RETENTION_BYTES
          value: "1073741824"
        - name: KAFKA_CFG_LOG_RETENTION_CHECK_INTERVALS_MS
          value: "300000"
        - name: KAFKA_CFG_LOG_RETENTION_HOURS
          value: "168"
        - name: KAFKA_CFG_LOG_MESSAGE_FORMAT_VERSION
        - name: KAFKA_CFG_MESSAGE_MAX_BYTES
          value: "1000012"
        - name: KAFKA_CFG_LOG_SEGMENT_BYTES
          value: "1073741824"
        - name: KAFKA_CFG_LOG_DIRS
          value: /bitnami/kafka/data
        - name: KAFKA_CFG_DEFAULT_REPLICATION_FACTOR
          value: "1"
        - name: KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR
          value: "1"
        - name: KAFKA_CFG_TRANSACTION_STATE_LOG_REPLICATION_FACTOR
          value: "1"
        - name: KAFKA_CFG_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM
          value: https
        - name: KAFKA_CFG_TRANSACTION_STATE_LOG_MIN_ISR
          value: "1"
        - name: KAFKA_CFG_NUM_IO_THREADS
          value: "8"
        - name: KAFKA_CFG_NUM_NETWORK_THREADS
          value: "3"
        - name: KAFKA_CFG_NUM_PARTITIONS
          value: "1"
        - name: KAFKA_CFG_NUM_RECOVERY_THREADS_PER_DATA_DIR
          value: "1"
        - name: KAFKA_CFG_SOCKET_RECEIVE_BUFFER_BYTES
          value: "102400"
        - name: KAFKA_CFG_SOCKET_REQUEST_MAX_BYTES
          value: "104857600"
        - name: KAFKA_CFG_SOCKET_SEND_BUFFER_BYTES
          value: "102400"
        - name: KAFKA_CFG_ZOOKEEPER_CONNECTION_TIMEOUT_MS
          value: "6000"
        image: docker.io/bitnami/kafka:2.3.0-debian-9-r88
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 2
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          tcpSocket:
            port: kafka
          timeoutSeconds: 5
        name: kafka
        ports:
        - containerPort: 9092
          name: kafka
          protocol: TCP
        readinessProbe:
          failureThreshold: 6
          initialDelaySeconds: 5
          periodSeconds: 10
          successThreshold: 1
          tcpSocket:
            port: kafka
          timeoutSeconds: 5
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /bitnami/kafka
          name: data
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 1001
        runAsUser: 1001
      terminationGracePeriodSeconds: 30
  updateStrategy:
    type: RollingUpdate
  volumeClaimTemplates:
  - metadata:
      creationTimestamp: null
      name: data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 8Gi
      volumeMode: Filesystem
    status:
      phase: Pending
status:
  collisionCount: 0
  currentReplicas: 3
  currentRevision: kafka-56ff499d74
  observedGeneration: 1
  readyReplicas: 1
  replicas: 3
  updateRevision: kafka-56ff499d74
  updatedReplicas: 3

我看到您在一个StatefulSet中为不同的pod传递不同的环境变量时遇到了一些问题

您正在尝试使用头盔模板实现此目标：

- name: KAFKA_CFG_ADVERTISED_LISTENERS
  {{- if $MY_POD_NAME := "kafka-0" }}
  value: 'PLAINTEXT://{{ .Values.advertisedListeners1 }}:9092'
  {{- else if $MY_POD_NAME := "kafka-1" }}
  value: 'PLAINTEXT://{{ .Values.advertisedListeners2 }}:9092'
  {{- else if $MY_POD_NAME := "kafka-2" }}
  value: 'PLAINTEXT://{{ .Values.advertisedListeners3 }}:9092'
  {{- end }}

在中，您可以找到以下解释：

在头盔模板中，变量是对另一个对象的命名引用。
它遵循$name的格式。变量使用特殊的赋值运算符进行赋值：：=

现在让我们看看您的代码：

{{- if $MY_POD_NAME := "kafka-0" }}

这是变量赋值，而不是比较和排序
在此赋值之后，
if
语句将此表达式计算为
true
，这就是为什么在
STAVELESET
yaml
manifest您将其视为一个输出：

- name: KAFKA_CFG_ADVERTISED_LISTENERS
    value: PLAINTEXT://10.21.0.191:9092


要使其按预期工作，您不应该使用头盔模板。这是行不通的

一种方法是为每个kafka节点创建单独的环境变量
并将所有这些变量传递给所有POD，如下所示：

- env:
  - name: MY_POD_NAME
    valueFrom:
      fieldRef:
        apiVersion: v1
        fieldPath: metadata.name
  - name: KAFKA_0
      value: 10.21.0.191
  - name: KAFKA_1
      value: 10.21.0.192
  - name: KAFKA_2
      value: 10.21.0.193
#  - name: KAFKA_CFG_ADVERTISED_LISTENERS
#      value: PLAINTEXT://$MY_POD_NAME:9092

apiVersion: v1
kind: Pod
metadata:
  name: test
spec:
  containers:
    - name: test-container
      image: docker.io/bitnami/kafka:2.3.0-debian-9-r88
      volumeMounts:
      - name: config-volume
        mountPath: /entrypoint.sh
        subPath: entrypoint.sh
  volumes:
    - name: config-volume
      configMap:
        # Provide the name of the ConfigMap containing the files you want
        # to add to the container
        name: kafka-entrypoint-config
        defaultMode: 0744 # remember to add proper (executable) permissions

apiVersion: v1
kind: ConfigMap
metadata:
  name: kafka-entrypoint-config
  namespace: default
data:
  entrypoint.sh: |
    #!/bin/bash
    # Here add modified entrypoint script

还可以使用修改后的起始脚本创建自己的docker映像，该脚本将导出
KAFKA\u CFG\u advised\u监听器
变量
根据
MY\u POD\u NAME
使用适当的值

如果不想创建自己的映像，可以使用修改后的
entrypoint.sh
创建一个
ConfigMap
，然后挂载它
代替旧的
entrypoint.sh
（您也可以使用任何其他文件，只需查看一下
有关如何构建卡夫卡图像的更多信息）

装载
ConfigMap
如下所示：

- env:
  - name: MY_POD_NAME
    valueFrom:
      fieldRef:
        apiVersion: v1
        fieldPath: metadata.name
  - name: KAFKA_0
      value: 10.21.0.191
  - name: KAFKA_1
      value: 10.21.0.192
  - name: KAFKA_2
      value: 10.21.0.193
#  - name: KAFKA_CFG_ADVERTISED_LISTENERS
#      value: PLAINTEXT://$MY_POD_NAME:9092

apiVersion: v1
kind: Pod
metadata:
  name: test
spec:
  containers:
    - name: test-container
      image: docker.io/bitnami/kafka:2.3.0-debian-9-r88
      volumeMounts:
      - name: config-volume
        mountPath: /entrypoint.sh
        subPath: entrypoint.sh
  volumes:
    - name: config-volume
      configMap:
        # Provide the name of the ConfigMap containing the files you want
        # to add to the container
        name: kafka-entrypoint-config
        defaultMode: 0744 # remember to add proper (executable) permissions

apiVersion: v1
kind: ConfigMap
metadata:
  name: kafka-entrypoint-config
  namespace: default
data:
  entrypoint.sh: |
    #!/bin/bash
    # Here add modified entrypoint script

如果有帮助，请告诉我。
这本指南正在谈论类似的事情，我想这对你来说可能很有趣：或者几天前打开的GitHub问题：@Nikhil你有机会查看卡洛斯的资料吗？@CarlosRodríguezHernández我查看了你共享的媒体博客。首先是孵化器/卡夫卡头盔图表。我需要让比特纳米/卡夫卡发挥作用。此外，我还尝试应用孵化器/卡夫卡所应用的更改，但它不起作用。@Nikhil能否更新答案并将
kubectl get statefulset-oyaml
的输出添加到其中？@HelloWorld我已更新了输出。。