Python 如何加快数据帧中的代码搜索速度需要几个小时

我有一个CSV文件，其中包含GIS模型中质心之间的距离，格式如下：

InputID,TargetID,Distance
1,2,3050.01327866
1,7,3334.99565217
1,5,3390.99115304
1,3,3613.77046864
1,4,4182.29900892
...
...
3330,3322,955927.582933

它按原点（

InputID

）排序，然后按最近的目的地（

TargetID

）排序

对于特定的建模工具，我需要CSV文件中的数据，格式如下（数字为质心数字）：

因此，没有InputID或TargetID，只有行上的起点和列上的目的地的距离： （前5个出发地/目的地示例）

我已经构建了下一个代码，它可以正常工作。但是它太慢了，运行它需要几天才能获得3330x3330文件。由于我是Python的初学者，我想我忽略了一些东西

import pandas as pd
import numpy as np
file=pd.read_csv('c:\\users\\Niels\\Dropbox\\Python\\centroid_distances.csv')
df=file.sort_index(by=['InputID', 'TargetID'], ascending=[True, True])
number_of_zones=3330
text_file = open("c:\\users\\Niels\\Dropbox\\Python\\Output.csv", "w")

for origin in range(1,number_of_zones):
    output_string=''
    print(origin)
    for destination in range(1,number_of_zones):
        if origin==destination:
            distance=0
        else:
            distance_row=df[(df['InputID']==origin) & (df['TargetID'] == destination)] 
            # I guess this is the time-consuming part
            distance=distance_row.iloc[0]['Distance']
        output_string=output_string+str(distance)+','
    text_file.write(output_string[:-1]+'\n') #strip last ',' of line
text_file.close()

---

你能给我一些提示来加速这个代码吗？

IIUC，你所需要的就是

pivot

。如果从这样的帧开始：

df = pd.DataFrame(columns="InputID,TargetID,Distance".split(","))
df["InputID"] = np.arange(36)//6 + 1
df["TargetID"] = np.arange(36) % 6 + 1
df["Distance"] = np.random.uniform(0, 100, len(df))
df = df[df.InputID != df.TargetID]
df = df.sort(["InputID", "Distance"])

>>> df.head()
   InputID  TargetID   Distance
2        1         3   6.407198
3        1         4  43.037829
1        1         2  52.121284
4        1         5  86.769620
5        1         6  96.703294

我们知道InputID和TargetID是唯一的，我们可以简单地

pivot

：

>>> pv = df.pivot(index="InputID", columns="TargetID", values="Distance").fillna(0)
>>> pv
TargetID          1          2          3          4          5          6
InputID                                                                   
1          0.000000  52.121284   6.407198  43.037829  86.769620  96.703294
2         53.741611   0.000000  27.555296  85.328607  59.561345   8.895407
3         96.142920  62.532984   0.000000   6.320273  37.809105  69.896308
4         57.835249  49.350647  38.660269   0.000000   7.151053  45.017780
5         72.758342  48.947788   4.212775  98.183169   0.000000  15.702280
6         32.468329  83.979431  23.578347  30.212883  82.580496   0.000000
>>> pv.to_csv("out_dist.csv", index=False, header=False)
>>> !cat out_dist.csv
0.0,52.1212839519,6.40719759732,43.0378290605,86.769620064,96.7032941473
53.7416111725,0.0,27.5552964592,85.3286070586,59.5613449796,8.89540736892
96.1429198049,62.5329836475,0.0,6.32027280686,37.8091052942,69.8963084944
57.8352492462,49.3506467609,38.6602692461,0.0,7.15105257546,45.0177800391
72.7583417281,48.9477878574,4.21277494476,98.183168992,0.0,15.7022798801
32.4683285321,83.9794307564,23.578346756,30.2128827937,82.5804959193,0.0

---

本教程的这一部分可能很有用。

我不懂CSV格式。您能提供一个明确的输入/输出示例吗？完成。提前谢谢！FWIW我对您大小的数据集（有3330个ID）做了与上面相同的事情，该数据集有更多的行，透视操作仍然在9秒内完成。赞许帝斯曼的枢轴解决方案

df = pd.DataFrame(columns="InputID,TargetID,Distance".split(","))
df["InputID"] = np.arange(36)//6 + 1
df["TargetID"] = np.arange(36) % 6 + 1
df["Distance"] = np.random.uniform(0, 100, len(df))
df = df[df.InputID != df.TargetID]
df = df.sort(["InputID", "Distance"])

>>> df.head()
   InputID  TargetID   Distance
2        1         3   6.407198
3        1         4  43.037829
1        1         2  52.121284
4        1         5  86.769620
5        1         6  96.703294

>>> pv = df.pivot(index="InputID", columns="TargetID", values="Distance").fillna(0)
>>> pv
TargetID          1          2          3          4          5          6
InputID                                                                   
1          0.000000  52.121284   6.407198  43.037829  86.769620  96.703294
2         53.741611   0.000000  27.555296  85.328607  59.561345   8.895407
3         96.142920  62.532984   0.000000   6.320273  37.809105  69.896308
4         57.835249  49.350647  38.660269   0.000000   7.151053  45.017780
5         72.758342  48.947788   4.212775  98.183169   0.000000  15.702280
6         32.468329  83.979431  23.578347  30.212883  82.580496   0.000000
>>> pv.to_csv("out_dist.csv", index=False, header=False)
>>> !cat out_dist.csv
0.0,52.1212839519,6.40719759732,43.0378290605,86.769620064,96.7032941473
53.7416111725,0.0,27.5552964592,85.3286070586,59.5613449796,8.89540736892
96.1429198049,62.5329836475,0.0,6.32027280686,37.8091052942,69.8963084944
57.8352492462,49.3506467609,38.6602692461,0.0,7.15105257546,45.0177800391
72.7583417281,48.9477878574,4.21277494476,98.183168992,0.0,15.7022798801
32.4683285321,83.9794307564,23.578346756,30.2128827937,82.5804959193,0.0