Python 如何并行化这个嵌套循环
我正在使用带有Dask的joblib来并行化具有以下循环结构的代码:Python 如何并行化这个嵌套循环,python,parallel-processing,dask,joblib,Python,Parallel Processing,Dask,Joblib,我正在使用带有Dask的joblib来并行化具有以下循环结构的代码: def main(): for semtype in semtypes: test = get_valid_systems(systems, semtype) expressions = get_ensemble_pairs(test) for c in expressions: <do stuff> 这很好用 现在,我想
def main():
for semtype in semtypes:
test = get_valid_systems(systems, semtype)
expressions = get_ensemble_pairs(test)
for c in expressions:
<do stuff>
这很好用
现在,我想添加两个循环,如中所示:
with joblib.parallel_backend('dask'):
joblib.Parallel(verbose=100)(joblib.delayed(main)(semtype, c) for c in get_ensemble_pairs(get_valid_systems(systems, semtype)) for semtype in semtypes)
但是,我得到一个错误,没有定义名称“semtype”。我假设这是我的
Paraallel
语句中函数调用的范围问题。我不太确定如何处理这个问题?最外层的循环应该首先出现
with joblib.parallel_backend('dask'):
joblib.Parallel(verbose=100)(joblib.delayed(main)(semtype, c) for semtype in semtypes for c in get_ensemble_pairs(get_valid_systems(systems, semtype)))
好吧,那太尴尬了!我是这样想的:
with joblib.parallel_backend('dask'):
joblib.Parallel(verbose=100)(joblib.delayed(main)(semtype, c) for semtype in semtypes for c in get_ensemble_pairs(get_valid_systems(systems, semtype)))