Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/309.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
python多处理星图vs apply_async,哪个更快?_Python_Pandas_Multiprocessing - Fatal编程技术网

python多处理星图vs apply_async,哪个更快?

python多处理星图vs apply_async,哪个更快?,python,pandas,multiprocessing,Python,Pandas,Multiprocessing,假设我有两种方法来完成相同的任务: from multiprocessing import Pool pool = Pool(4) def func(*args): # do some slow operations return something dates = ['2011-01-01', ' 2011-01-02', ... , '2017-01-01'] other_args = [1, 2, 3, 'c', 'test', 'pdf')] # approach 1

假设我有两种方法来完成相同的任务:

from multiprocessing import Pool
pool = Pool(4)

def func(*args):
    # do some slow operations
    return something

dates = ['2011-01-01', ' 2011-01-02', ... , '2017-01-01']
other_args = [1, 2, 3, 'c', 'test', 'pdf')]
# approach 1:
res = [pool.apply_async(func, [day] + other_args) for day in dates]
list_of_results = [x.get() for x in res]

# approach 2: create an iterable of iterables
args = [[day] + other_args for day in dates]
list_of_results = pool.starmap(func, args)

我意识到apply_async会立即返回,但是,如果func还没有完成运行,x.get()可能仍然会阻塞主线程……这两个方法之间必然会有性能差异吗

在引擎盖下,
星图
与第一次接近时所做的差不多。这只是一个方便的包装。提供的
map
函数系列符合许多开发人员习惯的函数编程范式

它们提供了一些很好的功能,比如将iterables分块以最小化IPC。性能优势可能来自此优化,但它取决于每个元素的计算成本


我建议坚持更具可读性的方法,并且只有当性能是真正的问题时,才对结果进行基准测试和评估

使用异步方法的关键是避免等待结果,因为它们将在以后使用。