Python 从DAG访问上下文
我想为DockerRoperator的volumes属性生成条目。我创建DockerRoperator的方式如下:Python 从DAG访问上下文,python,airflow,Python,Airflow,我想为DockerRoperator的volumes属性生成条目。我创建DockerRoperator的方式如下: def create_docker_operator(dag): # TODO: fill up volumes dynamically with data from REST paramters volumes = [ "{{ task_instance.xcom_pull(key='run_path') }}:/workdir" ]
def create_docker_operator(dag):
# TODO: fill up volumes dynamically with data from REST paramters
volumes = [ "{{ task_instance.xcom_pull(key='run_path') }}:/workdir" ]
return DockerOperator(task_id="start_docker",
container_name="{{ task_instance.xcom_pull(key='run_folder') }}_{{ task_instance.xcom_pull(key='project_name') }}",
image="{{ task_instance.xcom_pull(key='run_env_docker_name') }}",
api_version="auto",
auto_remove=False,
xcom_push=True,
xcom_all=True,
volumes=volumes,
command="python /workdir/bin/python_caller.py /workdir/bin/{{ task_instance.xcom_pull(key='script_name') }}",
network_mode="bridge",
dag=dag)
with airflow.DAG("learning", default_args=default_args, schedule_interval=None) as dag:
create_run_task = PythonOperator(task_id="create_run", provide_context=True, python_callable=create_run, dag=dag)
prepare_data_task = PythonOperator(task_id="prepare_data", provide_context=True, python_callable=prepare_data, dag=dag)
start_learning_dockered_task = create_docker_operator(dag)
check_result_task = PythonOperator(task_id="check_result", provide_context=True, python_callable=check_result, dag=dag)
create_run_task >> prepare_data_task >> start_learning_dockered_task >> check_result_task
DAG由传递一些参数的REST调用触发。在这个参数上,我希望通过迭代参数并将其附加到卷来动态填充卷。不幸的是,我没有这个解决方案的上下文来访问REST调用中的参数。是否可以从传递的dag实例访问它以创建\u docker\u operatordag