Python 查找多个值的最近索引_Python

Python 查找多个值的最近索引

python

Python 查找多个值的最近索引,python,Python,我正在尝试查找列中多个值的最近索引范围。我添加了一个python的最小工作代码，如下所示。我的数据范围比这个示例中的要大。它有3000多行。下面的操作和我预期的一样，但处理时间稍长，大约50-60秒我怎样才能减少这段时间？这种情况有不同的处理方法吗不是：我的X型和Y型VAL在“列表”和“VAL”列中，正面X VAL位于“观察”列中尝试使用FindNearest函数获取索引编号和计算的Y VAL import numpy as np import pandas as pd from dat

我正在尝试查找列中多个值的最近索引范围。我添加了一个python的最小工作代码，如下所示。我的数据范围比这个示例中的要大。它有3000多行。下面的操作和我预期的一样，但处理时间稍长，大约50-60秒

我怎样才能减少这段时间？这种情况有不同的处理方法吗

不是：我的X型和Y型VAL在“列表”和“VAL”列中，
正面X VAL位于“观察”列中
尝试使用FindNearest函数获取索引编号和计算的Y VAL

import numpy as np
import pandas as pd
from datetime import datetime as dt

def FindNearest(Table, Value):
    idx = Table['list'].sub(Value).abs().idxmin()       # find nearest index no
    row_nrst = Table.loc[idx]                   # get all vals at nearest

    # is value big (-1) or small (+1) from nearest, to decide second nearest val
    updwn = -1 if Value > row_nrst['list'] else 1

    # get model vals that value is between of them
    lst1, lval1 = row_nrst[['list', 'vals']]
    lst2, lval2 = Table.loc[idx+updwn, ['list', 'vals']]

    #calculate observed Y val
    rvals = lval1 + (lval2-lval1)*(Value-lst1)/(lst2-lst1)

    return pd.Series([idx, rvals])



start = dt.now()

aa = np.matrix([
    [  15,   14,  13,  12,   11,   10,   9,    8],              # model X vals
    [ 100.5, 94.5, 88.5, 66.5, 74.5, 91.5, 105.5, 120.5],       # model Y vals
    [12.3, 14.6, 8.7, 13.5, 14.2, 9.4, 11.3, 11.5],             # observed X vals
    [-1, -1, -1, -1, -1, -1, -1, -1],                           # index of model X vals
    [-1, -1, -1, -1, -1, -1, -1, -1]                            # calculalted observed Y vals
]).transpose()

tbl = pd.DataFrame(aa, columns=['list', 'vals', 'obser', 'ids', 'obsval'])

# finding process is peformed with **apply** function of Pandas library
tbl[['ids', 'obsval']] = tbl.apply(lambda x: FindNearest(tbl, x['obser']), axis=1)

elapsed = dt.now() - start

print(tbl)

print('Elapsed time :%2.3Fsn'%(elapsed.total_seconds()))

这里有一个方法，它对数据进行一次遍历，以创建具有最接近索引的列

这是您的原始代码

import pandas as pd
import numpy as np
aa = np.matrix([
    [  15,   14,  13,  12,   11,   10,   9,    8],              # model X vals
    [ 100.5, 94.5, 88.5, 66.5, 74.5, 91.5, 105.5, 120.5],       # model Y vals
    [12.3, 14.6, 8.7, 13.5, 14.2, 9.4, 11.3, 11.5],             # observed X vals
    [-1, -1, -1, -1, -1, -1, -1, -1],                           # index of model X vals
    [-1, -1, -1, -1, -1, -1, -1, -1]                            # calculated observed Y vals
]).transpose()

tbl = pd.DataFrame(aa, columns=['list', 'vals', 'obser', 'ids', 'obsval'])

然后我添加了两个哨兵行作为第一行和最后一行

bb = np.matrix([
    [  -999,   999],              # model X vals
    [ 0.0, 0.0],       # model Y vals
    [float ("-inf"), float("inf")],             # observed X vals
    [-1, -1],                           # index of model X vals
    [-1, -1]                            # calculated observed Y vals
]).transpose()
sentinel = pd.DataFrame(bb, columns=['list', 'vals', 'obser', 'ids', 'obsval'])
sentinel.head()

然后我创建了一个单独的数据框，它将您的数据与sentinel行组合在一起，进行排序

df = tbl.append(sentinel, ignore_index=False)
df.sort_values('obser', inplace=True)
print (df.head(10))

结果如下：

    list   vals  obser  ids  obsval
0 -999.0    0.0   -inf -1.0    -1.0
2   13.0   88.5    8.7 -1.0    -1.0
5   10.0   91.5    9.4 -1.0    -1.0
6    9.0  105.5   11.3 -1.0    -1.0
7    8.0  120.5   11.5 -1.0    -1.0
0   15.0  100.5   12.3 -1.0    -1.0
3   12.0   66.5   13.5 -1.0    -1.0
4   11.0   74.5   14.2 -1.0    -1.0
1   14.0   94.5   14.6 -1.0    -1.0
1  999.0    0.0    inf -1.0    -1.0

最后，创建三行：prev、curr和next。比较观察者并从两者中选择较小者

closest_idx = []

for i, row in enumerate(df[:2].itertuples()):
    if i == 0:
        print ('setting prev')
        prev = row
    if i == 1:
        print ('setting curr')
        curr = row


for i, next in enumerate(df[2:].itertuples()):
    if (next.obser - curr.obser) > (curr.obser - prev.obser):
        closer_idx = prev.Index
    else:
        closer_idx = next.Index

    print (f'for row {i}, using {closer_idx}')

    prev = curr
    curr = next

    closest_idx.append(closer_idx)

print (f'{closest_idx}')

这将打印出最近的索引行：

[5, 2, 7, 6, 7, 4, 1, 4]