启动python多处理池工作线程时发生ModuleNotFoundError
在我的项目中,我有一长串要定价的交易,所以我将其分成几块,并将它们委托给多处理池进行处理。假设包装类称为Valuation,方法是pool_calculate(),如下所示:启动python多处理池工作线程时发生ModuleNotFoundError,python,python-multiprocessing,python-module,Python,Python Multiprocessing,Python Module,在我的项目中,我有一长串要定价的交易,所以我将其分成几块,并将它们委托给多处理池进行处理。假设包装类称为Valuation,方法是pool_calculate(),如下所示: multiprocessing.freeze_support() p = multiprocessing.Pool() # default is a number of processes equal to the number of CPU cores results = p.map(self
multiprocessing.freeze_support()
p = multiprocessing.Pool() # default is a number of processes equal to the number of CPU cores
results = p.map(self.pooled_valuation_tasks, chunks)
p.close()
p.join()
问题是我们必须为所有定价导入一个内部模块,比如magic
import magic
class Valuation(object):
....
模块似乎没有传播到池工作线程中,错误消息如下所示:
Process SpawnPoolWorker-3:
Traceback (most recent call last):
File "c:\opt\magic0.0.46\PythonVenv\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "c:\opt\magic0.0.46\PythonVenv\lib\multiprocessing\process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "c:\opt\magic0.0.46\PythonVenv\lib\multiprocessing\pool.py", line 108, in worker
task = get()
File "c:\opt\magic0.0.46\PythonVenv\lib\multiprocessing\queues.py", line 337, in get
return _ForkingPickler.loads(res)
ModuleNotFoundError: No module named 'magic.rec'
有人能帮忙吗
更新:
针对评论中的一些问题提供更多细节:
这就是我实例化类并调用func的方式:
if __name__ == '__main__':
valu = Valuation(...)
valu.pool_calculate(...)
如果没有一个完整的例子来重现这个问题,很难回答这个问题。一些想法:您是否在模块的顶层导入
magic
?您是否使用保护创建多处理.Pool()
,如果uuuu name\uuuuuu==“\uuuuu main\uuuu”:
guard?