Airflow 气流池使用的插槽大于插槽限制
有三个传感器任务并使用同一个池,池“limit_Sensor”设置为1,但池限制不起作用,三个池同时运行Airflow 气流池使用的插槽大于插槽限制,airflow,Airflow,有三个传感器任务并使用同一个池,池“limit_Sensor”设置为1,但池限制不起作用,三个池同时运行 sensor_wait = SqlSensor( task_id='sensor_wait', dag=dag, conn_id='dest_data', sql="select count(*) from test", poke_interval=10, timeout=60, pool='limit_sensor', pri
sensor_wait = SqlSensor(
task_id='sensor_wait',
dag=dag,
conn_id='dest_data',
sql="select count(*) from test",
poke_interval=10,
timeout=60,
pool='limit_sensor',
priority_weight=100
)
same_pool1 = SqlSensor(
task_id='same_pool1',
dag=dag,
conn_id='dest_data',
sql="select count(*) from test",
poke_interval=10,
timeout=60,
pool='limit_sensor',
priority_weight=10
)
same_pool2 = SqlSensor(
task_id='same_pool2',
dag=dag,
conn_id='dest_data',
sql="select count(*) from test",
poke_interval=10,
timeout=60,
pool='limit_sensor',
priority_weight=10
)
有回填日志,气流1.10.0
[2018-10-12 11:20:35,036] {jobs.py:2198} INFO - [backfill progress] | finished run 0 of 1 | tasks waiting: 0 | succeeded: 0 | running: 3 | failed: 0 | skipped: 0 | deadlocked: 0 | not ready: 0
在web uiAdmin->Pools
中,您可以看到:
Pool Slots Used Slots Queued Slots
limit_sensor 1 3 0
我应该如何使池限制生效?谢谢