我在KubernetesPodOperator的DAG设置中做错了什么

我在KubernetesPodOperator的DAG设置中做错了什么,kubernetes,airflow,Kubernetes,Airflow,我在这里发现了以下气流DAG: 在我尝试添加任何自定义内容之前。。。试图按原样运行它。但是,在我的环境中,代码似乎超时 根据文档,我试图将启动\u超时\u秒设置为类似10m的荒谬值。。。但仍然收到文档中描述的超时消息: [2019-01-04 11:13:33,360] {pod_launcher.py:112} INFO - Event: fail-7dd76b92 had an event of type Pending Traceback (most recent call last):

我在这里发现了以下气流DAG:

在我尝试添加任何自定义内容之前。。。试图按原样运行它。但是,在我的环境中,代码似乎超时

根据文档,我试图将
启动\u超时\u秒
设置为类似10m的荒谬值。。。但仍然收到文档中描述的超时消息:

[2019-01-04 11:13:33,360] {pod_launcher.py:112} INFO - Event: fail-7dd76b92 had an event of type Pending
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/usr/local/lib/airflow/airflow/bin/airflow", line 27, in <module>
    args.func(args)
  File "/usr/local/lib/airflow/airflow/bin/cli.py", line 392, in run
    pool=args.pool,
  File "/usr/local/lib/airflow/airflow/utils/db.py", line 50, in wrapper
    result = func(*args, **kwargs)
  File "/usr/local/lib/airflow/airflow/models.py", line 1492, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/usr/local/lib/airflow/airflow/contrib/operators/kubernetes_pod_operator.py", line 123, in execute
    raise AirflowException('Pod Launching failed: {error}'.format(error=ex))
airflow.exceptions.AirflowException: Pod Launching failed: Pod took too long to start
[2019-01-04 11:13:33360]{pod_launcher.py:112}INFO-事件:fail-7dd76b92有一个挂起类型的事件
回溯(最近一次呼叫最后一次):
文件“/usr/local/bin/afflow”,第6行,在
exec(编译(打开(\uuuuu文件\uuuuu).read(),\uuuuu文件\uuuuuuuu,'exec'))
文件“/usr/local/lib/afflow/afflow/bin/afflow”,第27行,in
args.func(args)
文件“/usr/local/lib/afflow/afflow/bin/cli.py”,第392行,运行中
pool=args.pool,
文件“/usr/local/lib/afflow/afflow/utils/db.py”,第50行,在包装器中
结果=函数(*args,**kwargs)
文件“/usr/local/lib/aiffair/aiffair/models.py”,第1492行,在原始任务中
结果=任务\复制.执行(上下文=上下文)
文件“/usr/local/lib/aiffair/aiffair/contrib/operators/kubernetes_pod_operator.py”,执行中的第123行
raise AirflowException('Pod启动失败:{error}'。格式(error=ex))
airflow.Exception.AirflowException:吊舱启动失败:吊舱启动时间过长

任何输入都将不胜感激。

由于此代码未使用完全限定的图像,这意味着Airflow正在从中提取图像,
“Python:3.6”
“ubuntu:1604”
不适用于或在中的docker图像名称

“Python”命令也不应该大写

具有有效docker图像名称的工作代码为:

from airflow import DAG
from datetime import datetime, timedelta
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.operators.dummy_operator import DummyOperator


default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime.utcnow(),
    'email': ['airflow@example.com'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5)
}

dag = DAG(
    'kubernetes_sample', default_args=default_args, schedule_interval=timedelta(minutes=10))


start = DummyOperator(task_id='run_this_first', dag=dag)

passing = KubernetesPodOperator(namespace='default',
                          image="python:3.6-stretch",
                          cmds=["python","-c"],
                          arguments=["print('hello world')"],
                          labels={"foo": "bar"},
                          name="passing-test",
                          task_id="passing-task",
                          get_logs=True,
                          dag=dag
                          )

failing = KubernetesPodOperator(namespace='default',
                          image="ubuntu:16.04",
                          cmds=["python","-c"],
                          arguments=["print('hello world')"],
                          labels={"foo": "bar"},
                          name="fail",
                          task_id="failing-task",
                          get_logs=True,
                          dag=dag
                          )

passing.set_upstream(start)
failing.set_upstream(start)

是的,我是处理你案件的GCP支持人员;)我在这里也作了完整的回答(我知道,泰;)遗憾的是,没有日志记录可以帮助调试此类错误<代码>Pod启动时间太长帮不了什么忙…@LondonRob-同意。您知道如何在使用KubernetesPodOperator时获取更详细的日志吗?
from airflow import DAG
from datetime import datetime, timedelta
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.operators.dummy_operator import DummyOperator


default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime.utcnow(),
    'email': ['airflow@example.com'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5)
}

dag = DAG(
    'kubernetes_sample', default_args=default_args, schedule_interval=timedelta(minutes=10))


start = DummyOperator(task_id='run_this_first', dag=dag)

passing = KubernetesPodOperator(namespace='default',
                          image="python:3.6-stretch",
                          cmds=["python","-c"],
                          arguments=["print('hello world')"],
                          labels={"foo": "bar"},
                          name="passing-test",
                          task_id="passing-task",
                          get_logs=True,
                          dag=dag
                          )

failing = KubernetesPodOperator(namespace='default',
                          image="ubuntu:16.04",
                          cmds=["python","-c"],
                          arguments=["print('hello world')"],
                          labels={"foo": "bar"},
                          name="fail",
                          task_id="failing-task",
                          get_logs=True,
                          dag=dag
                          )

passing.set_upstream(start)
failing.set_upstream(start)