pandas中的python异步读取sql

pandas中的python异步读取sql,python,multithreading,pandas,Python,Multithreading,Pandas,我想通过将查询拆分为4来加快从数据库获取数据的过程。我使用apply\u async编写了以下代码。然而,在使用get时,出现了酸洗错误。我该怎么办?多谢各位 from multiprocessing import Pool pool = Pool(processes=4) start_date = datetime.datetime(2017, 1, 1) end_date = datetime.datetime(2017, 6, 30) period = (end_date-start_da

我想通过将查询拆分为4来加快从数据库获取数据的过程。我使用apply\u async编写了以下代码。然而,在使用get时,出现了酸洗错误。我该怎么办?多谢各位

from multiprocessing import Pool
pool = Pool(processes=4)
start_date = datetime.datetime(2017, 1, 1)
end_date = datetime.datetime(2017, 6, 30)
period = (end_date-start_date)/4
conn = pyodbc.connect(
    r'DRIVER={SQL Server};'
    r'SERVER=abc;'
    r'PORT=111;'
    r'DATABASE=db;'
    r'UID=abc;'
    r'PWD=xyz;'
    r'TDS_Version=7.1'
    )

for p in np.arange(start_date, end_date, period).astype(datetime.datetime):
    sql = "SELECT * FROM db where date between \'" +  str(p) +  "\' and \'" +  str(p + period) + "\'"
    res.append(pool.apply_async(lambda x: pd.read_sql(x[0], con = x[1]), ([sql, conn],)))      # runs in *only* one process
pool.close() 

res[0].get()#<-------PicklingError: Can't pickle <function <lambda> at 0x00000045566BDAE8>: attribute lookup <lambda>

您需要将连接线移动到每个子流程中:替换lambda x。。。通过一个例程连接到服务器,然后发送请求。不能打开单个连接并在子流程之间共享它

或者,您可以用aioodbc替换pyodbc:
这将允许您使用asyncio实现您需要的功能

有人能帮忙吗?您有没有弄明白这一点?