Python 为什么';dask是否并行化此工作流?
我有两个非常简单的函数:Python 为什么';dask是否并行化此工作流?,python,parallel-processing,dask,Python,Parallel Processing,Dask,我有两个非常简单的函数: import time def sleepy(a=1): time.sleep(a) print(a) def ending(*args): print(args) print('finished') workflow = {'task_0': (sleepy, 1), 'task_1': (sleepy, 2), 'task_2': (sleepy, 3),
import time
def sleepy(a=1):
time.sleep(a)
print(a)
def ending(*args):
print(args)
print('finished')
workflow = {'task_0': (sleepy, 1),
'task_1': (sleepy, 2),
'task_2': (sleepy, 3),
'ending': (ending, 'task_0', 'task_1', 'task_2')}
我还有一个使用以下功能的dask工作流:
import time
def sleepy(a=1):
time.sleep(a)
print(a)
def ending(*args):
print(args)
print('finished')
workflow = {'task_0': (sleepy, 1),
'task_1': (sleepy, 2),
'task_2': (sleepy, 3),
'ending': (ending, 'task_0', 'task_1', 'task_2')}
此工作流可以如下所示进行可视化:
sleepy
,sleepy
,sleepy
,应该是并行运行的,但它们不是
我等待1秒,它从sleepy()
打印1,然后我等待2秒,它打印2,然后我再等待3秒,它打印3:
1
2
3
(None, None, None)
finished
我做错了什么?这就是我编写您的工作流程的方式,睡眠操作确实是并行进行的
import dask.delayed
import time
@dask.delayed
def sleepy(a=1):
time.sleep(a)
print(a)
@dask.delayed
def ending(*args):
print(args)
print('finished')
d = ending(*[sleepy(i) for i in [1, 2, 3]])
d.compute()
请注意,
@
装饰程序只是语法上的精确性,您也可以执行dask.delayed(sleepy)
等操作。将dask.get(
更改为dask.threaded.get(
修复了我的问题,但我也非常喜欢mdurant的答案。您可以通过阅读上的第一个示例来实现。@user32185谢谢您,先生。