Docker 码头工人集装箱倒塌

我有3个容器，它们通过RabbitMQ发送作业，相互通信和交换数据。 从昨天开始，工作流中要执行的最后一个容器（

encoder\u aggregatore

）在构建完成后立即退出：

 $docker-compose up -d --build
  [..]
 $docker ps -a
 0f521c4260a5        encoder_my-pre-encoder   "python main.py"         About an hour ago   Up 20 minutes                                                                       encoder_my-pre-encoder_1
 72c3f5a2aa9c        encoder_aggregatore        "python /src/main.py"    18 hours ago        Exited (1) 20 minutes ago                                                           encoder_aggregatore_1
 dd9104e888ee        encoder_classificatore     "python /src/main.py"    20 hours ago        Up 20 minutes               0.0.0.0:8080->5000/tcp                                  encoder_classificatore_1
9

更多信息：

$docker inspect 72c3f5a2aa9c

输出为

我完全是个新手。我不知道这是否有帮助，但我注意到，如果我编辑“损坏”容器的代码，新代码将被编译，但会在容器

encoder\u my-pre-encoder

发送的“旧”数据作业上执行。我猜，由于某种原因，他们的交流中断了。 你能给我一些提示吗

编辑1：

docker-compose.yml：

version: '2.1'
services:
  files:
    image: busybox
    volumes:
      - ./file/input:/file/input
      - ./file/output:/file/output

  grafana:
    image: grafana/grafana:5.1.0
    ports:
      - 3000:3000
    volumes:
      - ./prometheus/grafana/:/etc/grafana/provisioning/
    environment:
      - GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-simple-json-datasource
      - GF_AUTH_BASIC_ENABLED=false
    depends_on: 
      - prometheus

  prometheus:
    image: prom/prometheus
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
    ports: 
      - 9090:9090

  aggregatore:
    build: aggregatore/.
    volumes:    
      - ./aggregatore:/src
    volumes_from: 
      - files
    ports:
      - 8000:8000
    command: ["python", "/src/main.py"]    
    depends_on: 
      rabbit:
        condition: service_healthy

  classificatore:
    build: classificatore/.
    volumes:    
      - ./classificatore:/src
    volumes_from: 
      - files
    ports: 
      - 8080:5000      
    command: ["python", "/src/main.py"]
    depends_on: 
      rabbit:
        condition: service_healthy

  my-pre-encoder:
    build: mpeg-pre-encoder/.
    volumes:
      - ./my-pre-encoder:/src
      - ./gabac_encoder:/src/gabac_encoder
    volumes_from: 
      - files    
    depends_on: 
      rabbit:
        condition: service_healthy

  rabbit:
    image: rabbitmq
    ports:
      - 5672:5672
    healthcheck:
      test: ["CMD", "rabbitmqctl", "cluster_status"]
      interval: 5s
      timeout: 2s
      retries: 20  

aggregatore容器日志：

Traceback (most recent call last):
  File "/src/main.py", line 94, in <module>
    rabbit.wait_for_job(worker)
  File "/src/util.py", line 75, in wait_for_job
    self.channel.start_consuming()
  File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 1822, in start_consuming
    self.connection.process_data_events(time_limit=None)
  File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 758, in process_data_events
    self._dispatch_channel_events()
  File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 521, in _dispatch_channel_events
    impl_channel._get_cookie()._dispatch_events()
  File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 1445, in _dispatch_events
    evt.body)
  File "/src/util.py", line 68, in callback
    worker_function(eval(body))
  File "<decorator-gen-1>", line 2, in worker
  File "/usr/local/lib/python3.7/site-packages/prometheus_client/context_managers.py", line 66, in wrapped
    return func(*args, **kwargs)
  File "/src/main.py", line 62, in worker
    updateMitb(out_file,  au['mitb_ptr'], au['AU_start_position'], au['AU_end_position'], au['dscn_value_ptr'])
KeyError: 'mitb_ptr'

日志 聚合矿/

$python main.py
2019-10-03 14:06:19,988 Could not get addresses to use: [Errno -2] Name or service not known (rabbit)
2019-10-03 14:06:19,988 Could not connect, 0 attempts left
2019-10-03 14:06:19,989 Connection open failed - gaierror(-2, 'Name or service not known')
Traceback (most recent call last):
  File "main.py", line 18, in <module>
    rabbit = Rabbit()
  File "/home/martina/PycharmProjects/mpegg_original/venv/lib/python3.6/site-packages/singleton_decorator/decorator.py", line 14, in __call__
    self._instance = self.__wrapped__(*args, **kwargs)
  File "/home/martina/PycharmProjects/mpegg_original/encoder/aggregatore/util.py", line 61, in __init__
    self.connection = pika.BlockingConnection(pika.ConnectionParameters(host=RABBIT_HOST))
  File "/home/martina/PycharmProjects/mpegg_original/venv/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 377, in __init__
    self._process_io_for_connection_setup()
  File "/home/martina/PycharmProjects/mpegg_original/venv/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 417, in _process_io_for_connection_setup
    self._open_error_result.is_ready)
  File "/home/martina/PycharmProjects/mpegg_original/venv/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 469, in _flush_output
    raise maybe_exception
  File "/home/martina/PycharmProjects/mpegg_original/venv/lib/python3.6/site-packages/pika/adapters/base_connection.py", line 176, in _adapter_connect
    socket.IPPROTO_TCP)
  File "/home/martina/PycharmProjects/mpegg_original/venv/lib/python3.6/site-packages/pika/adapters/base_connection.py", line 304, in _getaddrinfo
    return socket.getaddrinfo(host, port, family, socktype, proto)
  File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

$python main.py
2019-10-03 14:06:19988无法获取要使用的地址：[Errno-2]名称或服务未知（兔子）
2019-10-03 14:06:19988无法连接，还剩0次尝试
2019-10-03 14:06:19989连接打开失败-错误（-2，“名称或服务未知”）
回溯（最近一次呼叫最后一次）：
文件“main.py”，第18行，在
兔子
文件“/home/martina/PycharmProjects/mpegg_original/venv/lib/python3.6/site packages/singleton_decorator/decorator.py”，第14行，在调用中__
self.\u实例=self.\u包装的（*args，**kwargs）
文件“/home/martina/PycharmProjects/mpegg_-original/encoder/aggregatore/util.py”，第61行，在__
self.connection=pika.BlockingConnection（pika.ConnectionParameters（host=RABBIT\u host））
文件“/home/martina/PycharmProjects/mpegg_original/venv/lib/python3.6/site packages/pika/adapters/blocking_connection.py”，第377行，在__
self.\u进程\u io\u用于连接\u设置（）
文件“/home/martina/PycharmProjects/mpegg_original/venv/lib/python3.6/site packages/pika/adapters/blocking_connection.py”，第417行，进程中的连接设置
self.\u打开\u错误\u结果。准备好了吗）
文件“/home/martina/PycharmProjects/mpegg_original/venv/lib/python3.6/site packages/pika/adapters/blocking_connection.py”，第469行，在刷新输出中
提出可能的例外
文件“/home/martina/PycharmProjects/mpegg_original/venv/lib/python3.6/site packages/pika/adapters/base_connection.py”，第176行，在适配器连接中
socket.IPPROTO_（TCP）
文件“/home/martina/PycharmProjects/mpegg_original/venv/lib/python3.6/site packages/pika/adapters/base_connection.py”，第304行，在_getaddrinfo中
return socket.getaddrinfo（主机、端口、系列、socktype、proto）
文件“/usr/lib/python3.6/socket.py”，第745行，在getaddrinfo中
对于_socket.getaddrinfo（主机、端口、系列、类型、协议、标志）中的res：
socket.gaierror:[Errno-2]名称或服务未知

猛击到容器中：

$docker exec -it encoder_aggregatore_1 bash
Error response from daemon: Container 72c3f5a2aa9caba040bc60d1608c193e250967cde1a2b68a74d47cf36be887ee is not running

$docker run -it encoder_aggregatore /bin/bash

root@d5d030d88f76:/src# python main.py
2019-10-03 13:30:30,864 Connection to 31.199.53.9:5672 failed: timeout
2019-10-03 13:30:30,864 Could not connect, 0 attempts left
2019-10-03 13:30:30,865 Connection open failed - 'Connection to 31.199.53.9:5672 failed: timeout'
Traceback (most recent call last):
  File "main.py", line 18, in <module>
    rabbit = Rabbit()
  File "/usr/local/lib/python3.7/site-packages/singleton_decorator/decorator.py", line 14, in __call__
    self._instance = self.__wrapped__(*args, **kwargs)
  File "/src/util.py", line 61, in __init__
    self.connection = pika.BlockingConnection(pika.ConnectionParameters(host=RABBIT_HOST))
  File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 377, in __init__
    self._process_io_for_connection_setup()
  File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 417, in _process_io_for_connection_setup
    self._open_error_result.is_ready)
  File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 471, in _flush_output
    raise exceptions.ConnectionClosed(maybe_exception)
pika.exceptions.ConnectionClosed: Connection to 31.199.53.9:5672 failed: timeout

$docker exec-it编码器\u聚合器\u 1 bash
来自守护程序的错误响应：容器72C3F5A2AA9CABA040BC60D1608C193E250967CDE1A2B68A7447CF36BE887EE未运行
$docker运行-it编码器\u聚合器/bin/bash
root@d5d030d88f76：/src#python main.py
2019-10-03 13:30:30864连接到31.199.53.9:5672失败：超时
2019-10-03 13:30:30864无法连接，剩余0次尝试
2019-10-03 13:30:30865连接打开失败-“连接到31.199.53.9:5672失败：超时”
回溯（最近一次呼叫最后一次）：
文件“main.py”，第18行，在
兔子
文件“/usr/local/lib/python3.7/site packages/singleton\u decorator/decorator.py”，第14行，in\u调用__
self.\u实例=self.\u包装的（*args，**kwargs）
文件“/src/util.py”，第61行，在__
self.connection=pika.BlockingConnection（pika.ConnectionParameters（host=RABBIT\u host））
文件“/usr/local/lib/python3.7/site packages/pika/adapters/blocking_connection.py”，第377行，在__
self.\u进程\u io\u用于连接\u设置（）
文件“/usr/local/lib/python3.7/site packages/pika/adapters/blocking_connection.py”，第417行，用于连接设置的进程io
self.\u打开\u错误\u结果。准备好了吗）
文件“/usr/local/lib/python3.7/site packages/pika/adapters/blocking\u connection.py”，第471行，在\u flush\u输出中
引发异常。连接已关闭（可能是\u异常）
pika.exceptions.ConnectionClosed:连接到31.199.53.9:5672失败：超时

您需要添加compose文件，即失败容器的日志。尝试手动启动失败的容器并查看其行为。尝试将bash放入该容器并运行其运行的命令，手动并查看执行此操作时出现的问题。从创建持续崩溃的容器的映像创建容器：

docker run-it encoder\u my-pre-encoder/bin/bash

，然后运行在该容器中运行的启动命令。希望这是有道理的。然后，您应该看到在容器启动时不断发生的错误。这应该可以帮助你解决这个问题。我的坏意思是

docker run-it编码器\u aggregatore/bin/bash

，因为这是不断崩溃的容器的图像。这样做，它应该从同一个映像打开一个全新的容器，然后您应该能够调试该问题。希望这有帮助。我没有足够的上下文来回答这个问题。您需要添加compose文件，失败容器的日志。尝试手动启动失败的容器并查看其行为。尝试将bash放入该容器并运行其运行的命令，手动并查看执行此操作时出现的问题。从创建持续崩溃的容器的映像创建容器：

docker run-it encoder\u my-pre-encoder/bin/bash

，然后运行在该容器中运行的启动命令。希望这是有道理的。然后，您应该看到在容器启动时不断发生的错误。这应该可以帮助你解决这个问题。我的坏意思是

docker run-it编码器\u aggregatore/bin/bash

，因为这是不断崩溃的容器的图像。这样做，它应该从同一个映像打开一个全新的容器，然后您应该能够调试该问题。希望这有帮助。我没有足够的上下文来解释

$docker exec -it encoder_aggregatore_1 bash
Error response from daemon: Container 72c3f5a2aa9caba040bc60d1608c193e250967cde1a2b68a74d47cf36be887ee is not running

$docker run -it encoder_aggregatore /bin/bash

root@d5d030d88f76:/src# python main.py
2019-10-03 13:30:30,864 Connection to 31.199.53.9:5672 failed: timeout
2019-10-03 13:30:30,864 Could not connect, 0 attempts left
2019-10-03 13:30:30,865 Connection open failed - 'Connection to 31.199.53.9:5672 failed: timeout'
Traceback (most recent call last):
  File "main.py", line 18, in <module>
    rabbit = Rabbit()
  File "/usr/local/lib/python3.7/site-packages/singleton_decorator/decorator.py", line 14, in __call__
    self._instance = self.__wrapped__(*args, **kwargs)
  File "/src/util.py", line 61, in __init__
    self.connection = pika.BlockingConnection(pika.ConnectionParameters(host=RABBIT_HOST))
  File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 377, in __init__
    self._process_io_for_connection_setup()
  File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 417, in _process_io_for_connection_setup
    self._open_error_result.is_ready)
  File "/usr/local/lib/python3.7/site-packages/pika/adapters/blocking_connection.py", line 471, in _flush_output
    raise exceptions.ConnectionClosed(maybe_exception)
pika.exceptions.ConnectionClosed: Connection to 31.199.53.9:5672 failed: timeout