Airflow 计划的DAG未运行。如何诊断问题?

Airflow 计划的DAG未运行。如何诊断问题?,airflow,Airflow,我使用的是气流1.8.1。我有一个DAG,我相信我已经计划每5分钟运行一次,但它没有这样做: 忽略手动触发的2次成功的DAG运行 我查看该DAG的调度程序日志,发现: [2019-04-26 22:03:35,601] {jobs.py:343} DagFileProcessor839 INFO - Started process (PID=5653) to work on /usr/local/airflow/dags/retrieve_airflow_artifacts.py [2019-

我使用的是气流1.8.1。我有一个DAG,我相信我已经计划每5分钟运行一次,但它没有这样做:

忽略手动触发的2次成功的DAG运行

我查看该DAG的调度程序日志,发现:

[2019-04-26 22:03:35,601] {jobs.py:343} DagFileProcessor839 INFO - Started process (PID=5653) to work on /usr/local/airflow/dags/retrieve_airflow_artifacts.py
[2019-04-26 22:03:35,606] {jobs.py:1525} DagFileProcessor839 INFO - Processing file /usr/local/airflow/dags/retrieve_airflow_artifacts.py for tasks to queue
[2019-04-26 22:03:35,607] {models.py:168} DagFileProcessor839 INFO - Filling up the DagBag from /usr/local/airflow/dags/retrieve_airflow_artifacts.py
[2019-04-26 22:03:36,083] {jobs.py:1539} DagFileProcessor839 INFO - DAG(s) ['retrieve_airflow_artifacts'] retrieved from /usr/local/airflow/dags/retrieve_airflow_artifacts.py
[2019-04-26 22:03:36,112] {jobs.py:1172} DagFileProcessor839 INFO - Processing retrieve_airflow_artifacts
[2019-04-26 22:03:36,126] {jobs.py:566} DagFileProcessor839 INFO - Skipping SLA check for <DAG: retrieve_airflow_artifacts> because no tasks in DAG have SLAs
[2019-04-26 22:03:36,132] {models.py:323} DagFileProcessor839 INFO - Finding 'running' jobs without a recent heartbeat
[2019-04-26 22:03:36,132] {models.py:329} DagFileProcessor839 INFO - Failing jobs without heartbeat after 2019-04-26 21:58:36.132768
[2019-04-26 22:03:36,139] {jobs.py:351} DagFileProcessor839 INFO - Processing /usr/local/airflow/dags/retrieve_airflow_artifacts.py took 0.539 seconds
[2019-04-26 22:04:06,776] {jobs.py:343} DagFileProcessor845 INFO - Started process (PID=5678) to work on /usr/local/airflow/dags/retrieve_airflow_artifacts.py
[2019-04-26 22:04:06,780] {jobs.py:1525} DagFileProcessor845 INFO - Processing file /usr/local/airflow/dags/retrieve_airflow_artifacts.py for tasks to queue
[2019-04-26 22:04:06,780] {models.py:168} DagFileProcessor845 INFO - Filling up the DagBag from /usr/local/airflow/dags/retrieve_airflow_artifacts.py
[2019-04-26 22:04:07,258] {jobs.py:1539} DagFileProcessor845 INFO - DAG(s) ['retrieve_airflow_artifacts'] retrieved from /usr/local/airflow/dags/retrieve_airflow_artifacts.py
[2019-04-26 22:04:07,287] {jobs.py:1172} DagFileProcessor845 INFO - Processing retrieve_airflow_artifacts
[2019-04-26 22:04:07,301] {jobs.py:566} DagFileProcessor845 INFO - Skipping SLA check for <DAG: retrieve_airflow_artifacts> because no tasks in DAG have SLAs
[2019-04-26 22:04:07,307] {models.py:323} DagFileProcessor845 INFO - Finding 'running' jobs without a recent heartbeat
[2019-04-26 22:04:07,307] {models.py:329} DagFileProcessor845 INFO - Failing jobs without heartbeat after 2019-04-26 21:59:07.307607
[2019-04-26 22:04:07,314] {jobs.py:351} DagFileProcessor845 INFO - Processing /usr/local/airflow/dags/retrieve_airflow_artifacts.py took 0.538 seconds

有没有人能帮我弄清楚为什么我的DAG没有运行,因为我看起来有点高有点低,无法理解。

如果让我猜的话,我会说你的开始日期给你带来了一些问题

将args更改为静态启动,并防止其在过去的时间间隔内运行:

args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime(2019, 4, 27) #year month day
}
此外,为了便于阅读,请将DAG参数更改为相同的功能:

dag = DAG(
    dag_id='retrieve_airflow_artifacts', 
    default_args=args,
    schedule_interval="*/5 * * * *"
)
这应该允许调度程序拾取它

通常建议不要动态设置开始日期

摘自:

我们建议不要使用动态值作为开始日期,尤其是 datetime.now,因为它可能会很混乱。任务被触发 一旦周期结束,理论上,@小时DAG将永远无法获得 一个小时后,随着现在的发展


关于这一点的另一个问题是:

如果我不得不猜测的话,我会说你的开始日期给你带来了一些问题

将args更改为静态启动,并防止其在过去的时间间隔内运行:

args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime(2019, 4, 27) #year month day
}
此外,为了便于阅读,请将DAG参数更改为相同的功能:

dag = DAG(
    dag_id='retrieve_airflow_artifacts', 
    default_args=args,
    schedule_interval="*/5 * * * *"
)
这应该允许调度程序拾取它

通常建议不要动态设置开始日期

摘自:

我们建议不要使用动态值作为开始日期,尤其是 datetime.now,因为它可能会很混乱。任务被触发 一旦周期结束,理论上,@小时DAG将永远无法获得 一个小时后,随着现在的发展


关于这一点的另一个问题:

谢谢你,扎克,现在将根据这个进行调查。看起来这确实是个问题,谢谢你,扎克谢谢,扎克,现在将根据这个进行调查。看起来这确实是个问题,谢谢你,扎克