Airflow 子任务错误未失败

也许有人能告诉我我做错了什么。我在Airflow中有一个运行命令的任务，在日志中我得到以下错误：

[2018-05-30 11:22:43,814] {models.py:1428} INFO - Executing <Task(PythonOperator): computer_unload_and_load> on 2018-05-30 15:22:41.595535
[2018-05-30 11:22:43,814] {base_task_runner.py:115} INFO - Running: ['bash', '-c', 'airflow run copy_kiosk_status computer_unload_and_load 2018-05-30T15:22:41.595535 --job_id 23 --raw -sd DAGS_FOLDER/copy_poc.py']
[2018-05-30 11:22:44,367] {base_task_runner.py:98} INFO - Subtask: [2018-05-30 11:22:44,367] {__init__.py:45} INFO - Using executor SequentialExecutor
[2018-05-30 11:22:44,412] {base_task_runner.py:98} INFO - Subtask: [2018-05-30 11:22:44,412] {models.py:189} INFO - Filling up the DagBag from /other/airflow/dags/copy_kiosk.py
[2018-05-30 11:22:44,570] {cli.py:374} INFO - Running on host [redacted]
[2018-05-30 11:22:46,967] {base_task_runner.py:98} INFO - Subtask: ERROR:  schema "dw2" does not exist
[2018-05-30 11:22:46,969] {base_task_runner.py:98} INFO - Subtask: [2018-05-30 11:22:46,968] {python_operator.py:90} INFO - Done. Returned value was: None

下面是它正在调用的函数：

def load_table(host, db, schema, table, unload_task_id=False, file_path=False, **kwargs):
    """ load a csv file into a table
        if no file_path is given, uses XCOM to get the file_name returned by the unload task"""
    try:
        if not file_path:
            file_path = kwargs['ti'].xcom_pull(task_ids=unload_task_id)

        load_cmd = "\copy {}.{} FROM {} WITH (FORMAT CSV,  NULL '^',  HEADER)".format(schema, table, file_path)
        command = [ "psql",
                    "-U", "root",
                    "-h", host,
                    "-d", db,
                    "-c", load_cmd
                  ]
        subprocess.run(command)

    except:
        raise

---

显然，我知道如何修复错误（我在那里有错误的模式），但我想知道为什么任务成功而不是失败？我一定错过了什么明显的东西

您的代码示例看起来不错-不过有一件事。在此代码示例中，任务中缺少

provide\u context=True

除此之外，我认为这与处理错误的子流程有关。您可以尝试设置属性

check=True

，以便在出现问题时子流程将引发异常

或者，您可以检查子流程

check_returncode（）

，如果异常为非零，则抛出您自己的异常

以下是用于子流程的Python 3文档：

---

要单独尝试异常处理，最好创建一个任务，如果这样做不起作用，只引发一个异常。

我添加了更多日志和调用itThanks的任务，但没有为subprocess.run调用设置check=True。我非常专注于它是一个气流的东西，我没有意识到要检查实际运行的Python。

def load_table(host, db, schema, table, unload_task_id=False, file_path=False, **kwargs):
    """ load a csv file into a table
        if no file_path is given, uses XCOM to get the file_name returned by the unload task"""
    try:
        if not file_path:
            file_path = kwargs['ti'].xcom_pull(task_ids=unload_task_id)

        load_cmd = "\copy {}.{} FROM {} WITH (FORMAT CSV,  NULL '^',  HEADER)".format(schema, table, file_path)
        command = [ "psql",
                    "-U", "root",
                    "-h", host,
                    "-d", db,
                    "-c", load_cmd
                  ]
        subprocess.run(command)

    except:
        raise