Python Scipy-LAPACK函数是并行的吗?

Python Scipy-LAPACK函数是并行的吗?,python,parallel-processing,scipy,lapack,Python,Parallel Processing,Scipy,Lapack,我目前使用的是scipy.linalg.lapack.zheevd()函数,它在所有内核上运行,如果我尝试使用ProcessPoolExecutor()或ThreadPoolExecutor()从concurrent.futures将函数映射到参数数组,就会产生挂起和内存溢出 它使用了和我的测试系统一样多的内核,但我的印象是,由于GIL,Python中的事情通常不会并行化。这是使用OpenMP运行底层Fortran代码的结果吗 可以安全地假设这是并行的,并且不能进一步并行吗?这对于我的代码来说并

我目前使用的是
scipy.linalg.lapack.zheevd()
函数,它在所有内核上运行,如果我尝试使用
ProcessPoolExecutor()
ThreadPoolExecutor()
concurrent.futures
将函数映射到参数数组,就会产生挂起和内存溢出

它使用了和我的测试系统一样多的内核,但我的印象是,由于GIL,Python中的事情通常不会并行化。这是使用OpenMP运行底层Fortran代码的结果吗

可以安全地假设这是并行的,并且不能进一步并行吗?这对于我的代码来说并不是一个很大的瓶颈(找到400个唯一的1000x1000矩阵的本征系统;虽然可能需要放大,例如最终1000个2000x2000矩阵),但我正处于优化阶段

下面是一段有助于概念化的代码片段,但不代表实际的矩阵:

import numpy as np
from scipy import linalg as la
import concurrent.futures

# In real code
#         various parameters are used to build the matrix function,
#         it is presumably not sparse

# Matrix with independent variable x
def matrix_function(x):
    # Define dimensions and pre-allocate space for matrix
    #dim = 100        # For quicker evaluation/testing
    dim = 1000        # For conveying the scale of the problem
    matrix_dimensions = [dim, dim]
    # The matrix is complex
    mat = np.zeros(matrix_dimensions, dtype=complex)
    for i in range(dim):
        for j in range(i,dim):
            mat[i,j] = x*np.random.rand(1) + np.random.rand(1)*1J
            # Making the matrix Hermitian
            mat[j,i] = np.conjugate( mat[i,j] )
    return mat
        
# 400 Arguments for the defined matrix function
args = np.arange(0,10,0.025)

# Parallelizing evaluation of 400 matrices
with concurrent.futures.ProcessPoolExecutor() as pool:
    evaluated_matrix_functions = pool.map( matrix_function, args )
    ''' This will hang,
             which is what tipped me off to the issue
                                          **not important to question
        eigsystem = pool.map( la.lapack.zheevd,
                              evaluated_matrix_functions
                              )
    '''
    pool.shutdown()
    
''' This will cause a memory overflow,
              depending on the size of the matrices
              and how many of them; even with 32GB memory

with concurrent.futures.ThreadPoolExecutor() as pool:
    eigsystem = pool.map( la.lapack.zheevd,
                          evaluated_matrix_functions
                          )
    pool.shutdown()
'''

# The code which I run, in serial,
#          but still uses all cores/threads my 2700x provides at full load
eigensystem_list = []
for matrix in evaluated_matrix_functions:
    eigensystem_list.append( la.lapack.zheevd(matrix) )
    
# The eigensystem_list is then used in later calculations

这一切都由您在发动机罩下使用的LAPACK库控制。

这一切都由您在发动机罩下使用的LAPACK库控制