气流:DockerOperator失败,出现权限被拒绝错误

气流:DockerOperator失败,出现权限被拒绝错误,docker,airflow,Docker,Airflow,我试图通过Airflow运行docker容器,但获得的权限被拒绝错误。我看过一些相关的帖子,有些人似乎已经通过sudo chmod 777/var/run/docker.sock>解决了这个问题,这充其量是一个有问题的解决方案,但它对我仍然不起作用(即使在重启docker之后。如果有人设法解决了这个问题,请告诉我!) 这是我的DAG: from datetime import datetime, timedelta from airflow import DAG from airflow.ope

我试图通过Airflow运行docker容器,但获得的
权限被拒绝
错误。我看过一些相关的帖子,有些人似乎已经通过
sudo chmod 777/var/run/docker.sock>解决了这个问题,这充其量是一个有问题的解决方案,但它对我仍然不起作用(即使在重启docker之后。如果有人设法解决了这个问题,请告诉我!)

这是我的DAG:

from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.docker_operator import DockerOperator

args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime(2020, 6, 21, 11, 45, 0),
    'retries': 1,
    'retry_delay': timedelta(minutes=1),
}

dag = DAG(
    "docker",
    default_args=args,
    max_active_runs=1,
    schedule_interval='* * * * *',
    catchup=False
)

hello_operator = DockerOperator(
    task_id="run_docker",
    image="alpine:latest",
    command="/bin/bash echo HI!",
    auto_remove=True,
    dag=dag
)
下面是我得到的错误:

[2020-06-21 14:01:36,620] {taskinstance.py:1145} ERROR - ('Connection aborted.', PermissionError(13, 'Permission denied'))
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.6/http/client.py", line 1262, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1308, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1257, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/home/airflow/.local/lib/python3.6/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
PermissionError: [Errno 13] Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.6/http/client.py", line 1262, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1308, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1257, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/home/airflow/.local/lib/python3.6/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', PermissionError(13, 'Permission denied'))
以下是我的设置:

Dockerfile:

FROM apache/airflow
RUN pip install --upgrade --user pip && \
    pip install --user psycopg2-binary && \
    pip install --user docker
COPY airflow/airflow.cfg /opt/airflow/
docker-compose.yaml:

version: "3"

services:

  postgres:
    image: "postgres:9.6"
    container_name: "postgres"
    environment:
      - POSTGRES_USER=airflow
      - POSTGRES_PASSWORD=airflow
      - POSTGRES_DB=airflow
    ports:
    - "5432:5432"
    volumes:
    - ./data/postgres:/var/lib/postgresql/data

  initdb:
    image: learning/airflow
    entrypoint: airflow initdb
    depends_on:
      - postgres

  webserver:
    image: learning/airflow
    restart: always
    entrypoint: airflow webserver
    environment:
      - EXECUTOR=Local
    healthcheck:
      test: ["CMD-SHELL", "[ -f /opt/airflow/airflow-webserver.pid ]"]
      interval: 1m
      timeout: 5m
      retries: 3
    ports:
      - "8080:8080"
    depends_on:
      - postgres
    volumes:
    - ./airflow/dags:/opt/airflow/dags
    - ./airflow/plugins:/opt/airflow/plugins
    - ./data/logs:/opt/airflow/logs
    - /var/run/docker.sock:/var/run/docker.sock

  scheduler:
    image: learning/airflow
    restart: always
    entrypoint: airflow scheduler
    healthcheck:
      test: ["CMD-SHELL", "[ -f /opt/airflow/airflow-scheduler.pid ]"]
      interval: 1m
      timeout: 5m
      retries: 3
    depends_on:
      - postgres
    volumes:
      - ./airflow/dags:/opt/airflow/dags
      - ./airflow/plugins:/opt/airflow/plugins
      - ./data/logs:/opt/airflow/logs
      - /var/run/docker.sock:/var/run/docker.sock

中的
/var/run/docker.sock
中添加另一个前导的
/
(在
之前的源代码处:
),如下所示:

volumes:
    //var/run/docker.sock:/var/run/docker.sock

您可以尝试使用以下工具运行docker文件:

docker run -v /var/run/docker.sock:/var/run/docker.sock your_image_name

在我的例子中,命令帮助之前的“sudo”——我跑了

sudo docker-compose up -d --build dev
而不是

docker-compose up -d --build dev

这也有帮助。问题是缺乏权限。

我在windows(开发环境)上使用puckel映像遇到了这个问题。 请注意,此图像上不存在文件/var/run/docker.sock,我创建了它并将所有者更改为puckel图像中已存在的气流用户

RUN touch /var/run/docker.sock
RUN chown -R airflow /var/run/docker.sock

谢谢你的建议。不幸的是,它似乎没有帮助。我也尝试添加和额外/后冒号,但它也有帮助。这个“//”是Windows视窗更多的是没有描述,为什么这可能会帮助。考虑添加解释,如果你提供答案。