Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/327.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/clojure/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python Dagster:如何重新执行管道的失败步骤?_Python_Python 3.x_Pipeline_Dagster - Fatal编程技术网

Python Dagster:如何重新执行管道的失败步骤?

Python Dagster:如何重新执行管道的失败步骤?,python,python-3.x,pipeline,dagster,Python,Python 3.x,Pipeline,Dagster,我创建了一个测试管道,但中途失败了。我希望以编程方式重新执行它,但从管道的失败步骤开始并继续执行。我不想重复执行先前成功的步骤 from dagster import DagsterInstance, execute_pipeline, pipeline, solid, reexecute_pipeline from random import random instance = DagsterInstance.ephemeral() @solid def step1(context, da

我创建了一个测试管道,但中途失败了。我希望以编程方式重新执行它,但从管道的失败步骤开始并继续执行。我不想重复执行先前成功的步骤

from dagster import DagsterInstance, execute_pipeline, pipeline, solid, reexecute_pipeline
from random import random

instance = DagsterInstance.ephemeral()

@solid
def step1(context, data):
   return range(10), ('a' + i for i in range(10))

@solid
def step2(context, step1op):
  x,y = step1op

  # simulation of noise
  xx = [el * (1 + 0.1 * random()) for el in x]
  xx2 = [(el - 1)/el for el in xx]
  return zip(xx, xx2), y

@solid
def step3(context, step2op):
   x, y = step2op

   ...
   return x, y


run_config = {...}


@pipeline
def inputs_pipeline():
     step3(step2(step1()))

部分管道的编程重新执行需要识别母实体的ID,该ID可用:

parent_run_id = instance.get_runs()[0].run_id
然后,对管道进行再探测:

result = reexecute_pipeline(inputs_pipeline, parent_run_id=parent_run_id,
                            step_keys_to_execute=['step2.compute', 'step3.compute'],
                            run_config=run_config, instance=instance)

部分管道的编程重新执行需要识别母实体的ID,该ID可用:

parent_run_id = instance.get_runs()[0].run_id
然后,对管道进行再探测:

result = reexecute_pipeline(inputs_pipeline, parent_run_id=parent_run_id,
                            step_keys_to_execute=['step2.compute', 'step3.compute'],
                            run_config=run_config, instance=instance)