Airflow KeyError:';ti&x27;在ApacheXcom中

Airflow KeyError:';ti&x27;在ApacheXcom中,airflow,apache-airflow-xcom,Airflow,Apache Airflow Xcom,我们正在尝试运行一个带有2个任务的简单DAG,该任务将通过xcom进行数据通信 DAG文件: from __future__ import print_function import airflow from airflow import DAG from airflow.operators.bash_operator import BashOperator from airflow.operators.python_operator import PythonOperator args =

我们正在尝试运行一个带有2个任务的简单DAG,该任务将通过xcom进行数据通信

DAG文件:

from __future__ import print_function
import airflow
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators.python_operator import PythonOperator

args = {
    'owner': 'airflow',
    'start_date': airflow.utils.dates.days_ago(2)
}

dag = DAG(
    'example_xcom',
    schedule_interval="@once",
    default_args=args)

value_1 = [1, 2, 3]


def push(**kwargs):
    # pushes an XCom without a specific target
    kwargs['ti'].xcom_push(key='value from pusher 1', value=value_1)


def puller(**kwargs):
    ti = kwargs['ti']

    v1 = ti.xcom_pull(key=None, task_ids='push')
    assert v1 == value_1

    v1 = ti.xcom_pull(key=None, task_ids=['push'])
    assert (v1) == (value_1)


push1 = PythonOperator(
    task_id='push', dag=dag, python_callable=push)

pull = BashOperator(
    task_id='also_run_this',
    bash_command='echo {{ ti.xcom_pull(task_ids="push_by_returning") }}',
    dag=dag)

pull.set_upstream(push1)
但是在气流中运行DAG时,我们得到了以下异常

[2018-09-27 16:55:33,431] {base_task_runner.py:98} INFO - Subtask: [2018-09-27 16:55:33,430] {models.py:189} INFO - Filling up the DagBag from /home/airflow/gcs/dags/xcom.py
[2018-09-27 16:55:33,694] {base_task_runner.py:98} INFO - Subtask: Traceback (most recent call last):
[2018-09-27 16:55:33,694] {base_task_runner.py:98} INFO - Subtask:   File "/usr/local/bin/airflow", line 27, in <module>
[2018-09-27 16:55:33,696] {base_task_runner.py:98} INFO - Subtask:     args.func(args)
[2018-09-27 16:55:33,697] {base_task_runner.py:98} INFO - Subtask:   File "/usr/local/lib/python2.7/site-packages/airflow/bin/cli.py", line 392, in run
[2018-09-27 16:55:33,697] {base_task_runner.py:98} INFO - Subtask:     pool=args.pool,
[2018-09-27 16:55:33,698] {base_task_runner.py:98} INFO - Subtask:   File "/usr/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in wrapper
[2018-09-27 16:55:33,699] {base_task_runner.py:98} INFO - Subtask:     result = func(*args, **kwargs)
[2018-09-27 16:55:33,699] {base_task_runner.py:98} INFO - Subtask:   File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 1492, in _run_raw_task
[2018-09-27 16:55:33,701] {base_task_runner.py:98} INFO - Subtask:     result = task_copy.execute(context=context)
[2018-09-27 16:55:33,701] {base_task_runner.py:98} INFO - Subtask:   File "/usr/local/lib/python2.7/site-packages/airflow/operators/python_operator.py", line 89, in execute
[2018-09-27 16:55:33,702] {base_task_runner.py:98} INFO - Subtask:     return_value = self.execute_callable()
[2018-09-27 16:55:33,703] {base_task_runner.py:98} INFO - Subtask:   File "/usr/local/lib/python2.7/site-packages/airflow/operators/python_operator.py", line 94, in execute_callable
[2018-09-27 16:55:33,703] {base_task_runner.py:98} INFO - Subtask:     return self.python_callable(*self.op_args, **self.op_kwargs)
[2018-09-27 16:55:33,704] {base_task_runner.py:98} INFO - Subtask:   File "/home/airflow/gcs/dags/xcom.py", line 22, in push
[2018-09-27 16:55:33,707] {base_task_runner.py:98} INFO - Subtask:     kwargs['ti'].xcom_push(key='value from pusher 1', value=value_1)
[2018-09-27 16:55:33,708] {base_task_runner.py:98} INFO - Subtask: KeyError: 'ti'
[2018-09-27 16:55:33431]{base_task_runner.py:98}信息-子任务:[2018-09-27 16:55:33430]{models.py:189}信息-从/home/aiffair/gcs/dags/xcom.py填充行李
[2018-09-27 16:55:33694]{base_task_runner.py:98}信息-子任务:回溯(最近一次呼叫最后一次):
[2018-09-27 16:55:33694]{base_task_runner.py:98}信息-子任务:文件“/usr/local/bin/aiffair”,第27行,in
[2018-09-27 16:55:33696]{base_task_runner.py:98}信息-子任务:args.func(args)
[2018-09-27 16:55:33697]{base_task_runner.py:98}信息-子任务:文件“/usr/local/lib/python2.7/site packages/afflow/bin/cli.py”,第392行,运行中
[2018-09-27 16:55:33697]{base_task_runner.py:98}INFO-子任务:pool=args.pool,
[2018-09-27 16:55:33698]{base_task_runner.py:98}信息-子任务:文件“/usr/local/lib/python2.7/site packages/afflow/utils/db.py”,第50行,包装中
[2018-09-27 16:55:33699]{base_task_runner.py:98}信息-子任务:result=func(*args,**kwargs)
[2018-09-27 16:55:33699]{base_task_runner.py:98}信息-子任务:文件“/usr/local/lib/python2.7/site packages/afflow/models.py”,第1492行,在_run_raw_task中
[2018-09-27 16:55:33701]{base_task_runner.py:98}INFO-子任务:result=task_copy.execute(context=context)
[2018-09-27 16:55:33701]{base_task_runner.py:98}信息-子任务:文件“/usr/local/lib/python2.7/site packages/afflow/operators/python_operator.py”,执行中第89行
[2018-09-27 16:55:33702]{base_task_runner.py:98}INFO-子任务:return_value=self.execute_callable()
[2018-09-27 16:55:33703]{base_task_runner.py:98}信息-子任务:文件“/usr/local/lib/python2.7/site packages/afflow/operators/python_operator.py”,第94行,在execute_callable中
[2018-09-27 16:55:33703]{base_task_runner.py:98}信息-子任务:返回self.python可调用(*self.op_args,**self.op_kwargs)
[2018-09-27 16:55:33704]{base_task_runner.py:98}信息-子任务:文件“/home/aiffair/gcs/dags/xcom.py”,第22行,推送
[2018-09-27 16:55:33707]{base_task_runner.py:98}信息-子任务:kwargs['ti'].xcom_push(key='value from pusher 1',value=value_1)
[2018-09-27 16:55:33708]{base_task_runner.py:98}信息-子任务:keyrerror:'ti'

我们验证了DAG,但没有问题,请帮助我们解决此问题。

添加
为默认参数提供上下文:True
。这是定义**KWARG所需的

args = {
    'owner': 'airflow',
    'start_date': airflow.utils.dates.days_ago(2),
    'provide_context': True
}
提供上下文(bool)–如果设置为true,气流将传递一组可在函数中使用的关键字参数。这组KWARG与您可以在jinja模板中使用的内容完全对应。为此,您需要在函数头中定义**kwargs