将Kafka docker运行命令转换为Kubernetes
下面的Kubernetes yaml按预期创建资源,并且没有错误/重新启动。对kafka broker的外部请求在将Kafka docker运行命令转换为Kubernetes,docker,kubernetes,apache-kafka,apache-zookeeper,kubernetes-service,Docker,Kubernetes,Apache Kafka,Apache Zookeeper,Kubernetes Service,下面的Kubernetes yaml按预期创建资源,并且没有错误/重新启动。对kafka broker的外部请求在localhost:80005无限期挂起。我的理解是,docker的“网络”概念是通过服务在Kubernetes实现的。在本例中,我有一个NodePort服务,它将流量引入集群,然后是一个ClusterIp端口,它处理容器之间的通信。基于此Zookeeper/Kafka配置的非Kubernetes,docker run版本,我怀疑我的问题在于我的服务和/或播发的侦听器。互联网上充斥着
localhost:80005
无限期挂起。我的理解是,docker的“网络”概念是通过服务在Kubernetes实现的。在本例中,我有一个NodePort
服务,它将流量引入集群,然后是一个ClusterIp
端口,它处理容器之间的通信。基于此Zookeeper/Kafka配置的非Kubernetes,docker run
版本,我怀疑我的问题在于我的服务和/或播发的侦听器。互联网上充斥着docker compose文件和helm图表,但我似乎找不到一个有效的配置组合
卡夫卡/动物园管理员亚马尔:
# Zookeeper
apiVersion: apps/v1
kind: Deployment
metadata:
name: zookeeper-deploy
spec:
replicas: 1
selector:
matchLabels:
app: zookeeper-app
template:
metadata:
labels:
app: zookeeper-app
spec:
containers:
- name: zookeeper-app
image: confluentinc/cp-zookeeper:5.4.1
ports:
- containerPort: 2181
env:
- name: ZOOKEEPER_TICK_TIME
value: "2000"
- name: ZOOKEEPER_CLIENT_PORT
value: "2181"
---
apiVersion: v1
kind: Service
metadata:
name: zk-cluster-svc
labels:
app: zk
spec:
type: ClusterIP
selector:
app: zookeeper-app
ports:
- port: 2181
targetPort: 2181
protocol: TCP
---
# Kafka
apiVersion: apps/v1
kind: Deployment
metadata:
name: kafka-deploy
spec:
replicas: 1
selector:
matchLabels:
app: kafka-app
template:
metadata:
labels:
app: kafka-app
spec:
containers:
- name: kafka-app
image: confluentinc/cp-kafka:5.4.1
ports:
- containerPort: 9092
- containerPort: 29092
env:
- name: KAFKA_BROKER_ID
value: "1"
- name: KAFKA_ZOOKEEPER_CONNECT
value: "zk-cluster-svc:2181"
- name: KAFKA_ADVERTISED_LISTENERS
value: "INSIDE://kafka-cluster-svc:29092,OUTSIDE://localhost:30005"
- name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
value: "INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT"
- name: KAFKA_INTER_BROKER_LISTENER_NAME
value: "INSIDE"
- name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR
value: "1"
---
apiVersion: v1
kind: Service
metadata:
name: kafka-service
labels:
app: kafka-service
spec:
type: NodePort
ports:
- port: 9092
nodePort: 30005
protocol: TCP
selector:
app: kafka-app
---
apiVersion: v1
kind: Service
metadata:
name: kafka-cluster-svc
labels:
app: kafka-app
spec:
selector:
app: kafka-app
ports:
- port: 29092
targetPort: 29092
protocol: TCP
^这是基于这些容器的正常运行版本:
docker network create kafzoonet
是否为单个Kafka节点正确配置了服务?卡夫卡似乎能够联系到Zookeeper,但由于某种原因,当客户端连接时,它无法得到正确的侦听器回音。不幸的是,我没有任何错误消息来支持这一点,因为客户端挂起,Kafka或Zookeeper播客也不会记录任何内容。虽然这不是您所要求的,但尝试在kubernetes中手动配置Kafka和Zookeeper是浪费时间的——这是一个通过“为什么外部地址是本地主机”解决的问题?你没有使用合流舵图吗?
docker run --name app-zookeeper -d --network="kafzoonet" --env ZOOKEEPER_TICK_TIME=2000 --env ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:5.4.1
docker run --name app-kafka -p 30005:9092 -d --network="kafzoonet" --env KAFKA_BROKER_ID=1 --env KAFKA_ZOOKEEPER_CONNECT="app-zookeeper:2181" --env KAFKA_ADVERTISED_LISTENERS="PLAINTEXT://app-kafka:29092,PLAINTEXT_HOST://localhost:30005" --env KAFKA_LISTENER_SECURITY_PROTOCOL_MAP="PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT" --env KAFKA_INTER_BROKER_LISTENER_NAME=PLAINTEXT --env KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 confluentinc/cp-kafka:5.4.1