Python 在具有多个工人的scipy差分进化中，将参数传递给目标函数

对于一些优化问题，我使用scipys优化工具箱中的差分进化。 我想使用几个CPU来加速这个过程，但我想把几个额外的参数传递给目标函数。然而，这些不仅仅是一些标量，而是优化评估模型所需的一些数据集

当我试图以通常的方式将参数直接传递给目标函数时，python抱怨目标函数不可拾取。当我将数据放入字典并将其传递给目标函数时，python抱怨 “文件“/usr/lib64/python3.6/multiprocessing/connection.py”，第393行，以字节为单位 header=struct.pack（“！i”，n） struct.error:“i”格式需要-2147483648如果提供了a，它总是有帮助的。要使用并行处理，目标函数和参数需要是可拾取的

以下说明了问题：

-------stuff.py--------
from scipy.optimize import rosen

def obj(x, *args):
    return rosen(x)

-------in the CLI-------
import numpy as np
import pickle
from scipy.optimize import differential_evolution, rosen
from stuff import obj

train_data = "123"
par02 = {'a':2,'b':3, "data":train_data}

bounds = [(0, 10), (0, 10)]

# This will work because `obj` is importable in
# the __main__ context.
differential_evolution(obj, bounds, args=(par02,),
                       updating='deferred', workers=-1)


def some_other_func(x, *args):
    pass

wont_work = {'a':2,'b':3, "data":some_other_func}

# look at the output here. ` some_other_func` is referenced with
# respect to __main__
print(pickle.dumps(wont_work))

# the following line will hang because `some_other_func`
# is not importable by the main process; to get it
# to work the function has to reside in an importable file.
parallel_result = differential_evolution(obj, bounds, args=(wont_work,),
                                         updating='deferred', workers=-1)

基本上，您不能使用在main上下文（即CLI）中定义的类/函数，它们必须可由main导入

-------stuff.py--------
from scipy.optimize import rosen

def obj(x, *args):
    return rosen(x)

-------in the CLI-------
import numpy as np
import pickle
from scipy.optimize import differential_evolution, rosen
from stuff import obj

train_data = "123"
par02 = {'a':2,'b':3, "data":train_data}

bounds = [(0, 10), (0, 10)]

# This will work because `obj` is importable in
# the __main__ context.
differential_evolution(obj, bounds, args=(par02,),
                       updating='deferred', workers=-1)


def some_other_func(x, *args):
    pass

wont_work = {'a':2,'b':3, "data":some_other_func}

# look at the output here. ` some_other_func` is referenced with
# respect to __main__
print(pickle.dumps(wont_work))

# the following line will hang because `some_other_func`
# is not importable by the main process; to get it
# to work the function has to reside in an importable file.
parallel_result = differential_evolution(obj, bounds, args=(wont_work,),
                                         updating='deferred', workers=-1)