Apache storm 在Storm 0.10.0上,即使我将workers设置为1,并且UI报告workers=1,也会启动2个worker进程
我有一个风暴拓扑,我做:Apache storm 在Storm 0.10.0上,即使我将workers设置为1,并且UI报告workers=1,也会启动2个worker进程,apache-storm,Apache Storm,我有一个风暴拓扑,我做:setNumWorkers(1) 当我查看此运行拓扑上的storm UI报告时,我看到Num workers设置为1 但是,当我登录到运行supervisor的节点时,我看到两个进程对-Dworker.id和-Dworker.port具有相同的设置 我包括ps显示的以下两个过程的输出 我的问题是:如果我只请求了一个进程,为什么会有两个进程被配置为工作进程(注意:Storm UI确认我只有一个工作进程) 这对我来说很重要,因为当我对我的拓扑所消耗的资源进行任何分析或分析时,
setNumWorkers(1)代码>
当我查看此运行拓扑上的storm UI报告时,我看到Num workers设置为1
但是,当我登录到运行supervisor的节点时,我看到两个进程对-Dworker.id
和-Dworker.port
具有相同的设置
我包括ps
显示的以下两个过程的输出
我的问题是:如果我只请求了一个进程,为什么会有两个进程被配置为工作进程(注意:Storm UI确认我只有一个工作进程)
这对我来说很重要,因为当我对我的拓扑所消耗的资源进行任何分析或分析时,我想知道应该集中在哪个进程上
ps输出
为了防止阅读本文的人试图更好地了解我的环境,这里是我的Storm docker配置(以及其他内容)。希望这能有所帮助
version: '2'
services:
zookeeper:
image: wurstmeister/zookeeper
container_name: zk
hostname: zk
ports:
- "2181:2181"
networks:
storm:
kafka:
image: wurstmeister/kafka:0.8.2.2-1
container_name: kafka
hostname: kafka
ports:
- "9092:9092"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ADVERTISED_HOST_NAME: 10.211.55.4
KAFKA_ZOOKEEPER_CONNECT: 10.211.55.4
volumes:
- /var/run/docker.sock:/var/run/docker.sock
nimbus:
image: sunside/storm-nimbus
container_name: storm-nimbus
hostname: storm-nimbus
ports:
- "49773:49772"
- "49772:49773"
- "49627:49627"
environment:
- "LOCAL_HOSTNAME=nimbus"
- "ZOOKEEPER_ADDRESS=zk"
- "ZOOKEEPER_PORT=2181"
- "NIMBUS_ADDRESS=nimbus"
- "NIMBUS_THRIFT_PORT=49627"
- "DRPC_PORT=49772"
- "DRPCI_PORT=49773"
volumes:
- /media/psf/Home/dev/storm-pipeline:/pipeline
networks:
storm:
supervisor:
image: sunside/storm-supervisor
container_name: storm-supervisor
hostname: storm-supervisor
ports:
- "8000:8000"
environment:
- "LOCAL_HOSTNAME=supervisor"
- "NIMBUS_ADDRESS=nimbus"
- "NIMBUS_THRIFT_PORT=49627"
- "DRPC_PORT=49772"
- "DRPCI_PORT=49773"
- "ZOOKEEPER_ADDRESS=zk"
- "ZOOKEEPER_PORT=2181"
networks:
storm:
ui:
image: sunside/storm-ui
container_name: storm-ui
hostname: storm-ui
ports:
- "8888:8080"
environment:
- "LOCAL_HOSTNAME=ui"
- "NIMBUS_ADDRESS=nimbus"
- "NIMBUS_THRIFT_PORT=49627"
- "DRPC_PORT=49772"
- "DPRCI_PORT=49773"
- "ZOOKEEPER_ADDRESS=zk"
- "ZOOKEEPER_PORT=2181"
networks:
storm:
elasticsearch:
image: elasticsearch:2.3
container_name: elasticsearch
hostname: elasticsearch
ports:
- "9200:9200"
networks:
storm:
networks:
storm:
external: true
答案是这两个进程中有一个是真正的“worker”进程(正在执行的类是backtype.storm.daemon.worker)。。。响应“ps”命令打印出来的另一个进程是日志编写器进程,由类backtype.storm.LogWriter执行
我应该在这两个进程的输出行中注意到这一点。哦,好吧。。。。现在我们知道了 多亏了霍顿工厂的迪普纳·贝恩斯(Deepna Bains)的支持,我们发现了预期结果和我们得到的结果之间差异的原因。
version: '2'
services:
zookeeper:
image: wurstmeister/zookeeper
container_name: zk
hostname: zk
ports:
- "2181:2181"
networks:
storm:
kafka:
image: wurstmeister/kafka:0.8.2.2-1
container_name: kafka
hostname: kafka
ports:
- "9092:9092"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ADVERTISED_HOST_NAME: 10.211.55.4
KAFKA_ZOOKEEPER_CONNECT: 10.211.55.4
volumes:
- /var/run/docker.sock:/var/run/docker.sock
nimbus:
image: sunside/storm-nimbus
container_name: storm-nimbus
hostname: storm-nimbus
ports:
- "49773:49772"
- "49772:49773"
- "49627:49627"
environment:
- "LOCAL_HOSTNAME=nimbus"
- "ZOOKEEPER_ADDRESS=zk"
- "ZOOKEEPER_PORT=2181"
- "NIMBUS_ADDRESS=nimbus"
- "NIMBUS_THRIFT_PORT=49627"
- "DRPC_PORT=49772"
- "DRPCI_PORT=49773"
volumes:
- /media/psf/Home/dev/storm-pipeline:/pipeline
networks:
storm:
supervisor:
image: sunside/storm-supervisor
container_name: storm-supervisor
hostname: storm-supervisor
ports:
- "8000:8000"
environment:
- "LOCAL_HOSTNAME=supervisor"
- "NIMBUS_ADDRESS=nimbus"
- "NIMBUS_THRIFT_PORT=49627"
- "DRPC_PORT=49772"
- "DRPCI_PORT=49773"
- "ZOOKEEPER_ADDRESS=zk"
- "ZOOKEEPER_PORT=2181"
networks:
storm:
ui:
image: sunside/storm-ui
container_name: storm-ui
hostname: storm-ui
ports:
- "8888:8080"
environment:
- "LOCAL_HOSTNAME=ui"
- "NIMBUS_ADDRESS=nimbus"
- "NIMBUS_THRIFT_PORT=49627"
- "DRPC_PORT=49772"
- "DPRCI_PORT=49773"
- "ZOOKEEPER_ADDRESS=zk"
- "ZOOKEEPER_PORT=2181"
networks:
storm:
elasticsearch:
image: elasticsearch:2.3
container_name: elasticsearch
hostname: elasticsearch
ports:
- "9200:9200"
networks:
storm:
networks:
storm:
external: true