在Python中,如何在两个值之间选择数据帧中的行?

在Python中,如何在两个值之间选择数据帧中的行?,python,pandas,Python,Pandas,我试图修改数据帧df以仅包含列closing\u price中的值介于99和101之间的行,并尝试使用下面的代码执行此操作 然而,我得到了错误 ValueError:序列的真值不明确。使用a.empty、a.bool()、a.item()、a.any()或a.all() 我想知道是否有一种不用循环就能做到这一点的方法 df = df[(99 <= df['closing_price'] <= 101)] df=df[(99您应该使用()对布尔向量进行分组以消除歧义 df = df[

我试图修改数据帧
df
以仅包含列
closing\u price
中的值介于99和101之间的行,并尝试使用下面的代码执行此操作

然而,我得到了错误

ValueError:序列的真值不明确。使用a.empty、a.bool()、a.item()、a.any()或a.all()

我想知道是否有一种不用循环就能做到这一点的方法

df = df[(99 <= df['closing_price'] <= 101)]
df=df[(99您应该使用
()
对布尔向量进行分组以消除歧义

df = df[(df['closing_price'] >= 99) & (df['closing_price'] <= 101)]

df=df[(df['closing_price']>=99)和(df['closing_price']]有一个更好的选择-使用方法:

[58]中的
:df=pd.DataFrame({'closing_price':np.random.randint(95,105,10)})
In[59]:df
出[59]:
收盘价
0            104
1             99
2             98
3             95
4            103
5            101
6            101
7             99
8             95
9             96
在[60]:df.query('99中也考虑:

newdf=df.query('closing_price.mean()而不是这个

df = df[(99 <= df['closing_price'] <= 101)]

df=df[(99您也可以使用
.between()
方法

emp = pd.read_csv("C:\\py\\programs\\pandas_2\\pandas\\employees.csv")

emp[emp["Salary"].between(60000, 61000)]
输出


如果您处理的是多个值和多个输入,您也可以设置这样的应用功能。在这种情况下,过滤属于特定范围的GPS位置的数据帧

def filter_values(lat,lon):
    if abs(lat - 33.77) < .01 and abs(lon - -118.16) < .01:
        return True
    elif abs(lat - 37.79) < .01 and abs(lon - -122.39) < .01:
        return True
    else:
        return False


df = df[df.apply(lambda x: filter_values(x['lat'],x['lon']),axis=1)]
def过滤器_值(纬度、经度):
如果abs(lat-33.77)<.01和abs(lon--118.16)<.01:
返回真值
elif abs(lat-37.79)<.01和abs(lon--122.39)<.01:
返回真值
其他:
返回错误
df=df[df.apply(λx:filter_值(x['lat'],x['lon']),轴=1)]
如果必须重复调用(对于不同的边界
l
r
),则会重复大量不必要的工作。在这种情况下,对帧/序列进行一次排序然后使用是有益的。我测量的加速比高达25倍,见下文

def between_indices(x, lower, upper, inclusive=True):
    """
    Returns smallest and largest index i for which holds 
    lower <= x[i] <= upper, under the assumption that x is sorted.
    """
    i = x.searchsorted(lower, side="left" if inclusive else "right")
    j = x.searchsorted(upper, side="right" if inclusive else "left")
    return i, j

# Sort x once before repeated calls of between()
x = x.sort_values().reset_index(drop=True)
# x = x.sort_values(ignore_index=True) # for pandas>=1.0
ret1 = between_indices(x, lower=0.1, upper=0.9)
ret2 = between_indices(x, lower=0.2, upper=0.8)
ret3 = ...
将numpy导入为np
作为pd进口熊猫
指数之间的def(x,下,上,包括=真):
#假设:x已排序。
i=x.searchsorted(下,side=“left”如果包含,否则为“right”)
j=x.searchsorted(上,side=“right”如果包含,否则为“left”)
返回i,j
def介于_fast之间(x,下,上,包括=真):
"""
在假设x已排序的情况下,等价于pd.Series.between()。
"""
i、 j=指数之间(x,下,上,含)
如果为真:
返回x.iloc[i:j]
其他:
#遮罩创建很慢。
掩码=np.zeros_like(x,dtype=bool)
掩码[i:j]=True
掩码=pd系列(掩码,索引=x索引)
返回x[掩码]
def介于(x,下,上,包括=真)之间:
遮罩=x.between(下、上、含=含)
返回x[掩码]
def在_expr(x,下,上,包括=真)之间:
如果包括:
遮罩=(x>=较低)和(xlower)和(x=1.0:
#x=x.sort_值(忽略_索引=True)
x=x.排序值().重置索引(drop=True)
#断言不同方法的等价性。
断言(介于(x,0,1,真)和等于(x,0,1,真)之间)
断言(在(x,0,1,True)之间)。等于(在(x,0,1,True)之间)
断言(介于(x,0,1,False)和等于(x,0,1,False)之间)
断言(在(x,0,1,False)之间)。等于(在(x,0,1,False)之间)
#对()之间的重复评估进行基准测试。
uppers=np.linspace(0,3,n次)
下压=-上压
%timeit基准测试(在快速、x、下降、上升之间)
%timeit基准(介于、x、下、上)
%timeit基准(在expr、x、lowers和UPPER之间)

这里的问题是,您无法将标量与数组进行比较,因此会出现错误。由于运算符优先级
df,为了进行比较,您必须使用位运算符并将它们括在括号中。query
pd.eval
似乎非常适合此用例。有关
pd.eval()的信息
函数系列、它们的功能和用例,请访问。我喜欢这里的语法,但在尝试与expresison;df.query('(mean+2*sd)组合时,我失败了@mappingdom,什么是
mean
sd
?这些是列名吗?不,它们是存储为float@mappingdom,你说“存储”是什么意思?@ManojKumar,
df.query('closing_price.between(99101,inclusive=True)),engine=“python”)
-但这比“numexpr”要慢引擎。pandas中是否有“not between”功能?我找不到它。@dsugasa,使用with
between
@dsugasa,例如
df=df[~df[~df['closing_price'])。between(99101)]
这应该是答案。我们是否有可能在
中使用
.betwever()
?我很想知道。我想知道我们是否可以在
.query()中使用
.between()
??
mean = closing_price.mean()
std = closing_price.std()

newdf = df.query('@mean <= closing_price <= @std')
df = df[(99 <= df['closing_price'] <= 101)]
df = df[(df['closing_price']>=99 ) & (df['closing_price']<=101)]
emp = pd.read_csv("C:\\py\\programs\\pandas_2\\pandas\\employees.csv")

emp[emp["Salary"].between(60000, 61000)]
def filter_values(lat,lon):
    if abs(lat - 33.77) < .01 and abs(lon - -118.16) < .01:
        return True
    elif abs(lat - 37.79) < .01 and abs(lon - -122.39) < .01:
        return True
    else:
        return False


df = df[df.apply(lambda x: filter_values(x['lat'],x['lon']),axis=1)]
def between_indices(x, lower, upper, inclusive=True):
    """
    Returns smallest and largest index i for which holds 
    lower <= x[i] <= upper, under the assumption that x is sorted.
    """
    i = x.searchsorted(lower, side="left" if inclusive else "right")
    j = x.searchsorted(upper, side="right" if inclusive else "left")
    return i, j

# Sort x once before repeated calls of between()
x = x.sort_values().reset_index(drop=True)
# x = x.sort_values(ignore_index=True) # for pandas>=1.0
ret1 = between_indices(x, lower=0.1, upper=0.9)
ret2 = between_indices(x, lower=0.2, upper=0.8)
ret3 = ...
# pd.Series.searchsorted()
# 5.87 ms ± 321 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
# pd.Series.between(lower, upper)
# 155 ms ± 6.08 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
# Logical expressions: (x>=lower) & (x<=upper)
# 153 ms ± 3.52 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
import numpy as np
import pandas as pd

def between_indices(x, lower, upper, inclusive=True):
    # Assumption: x is sorted.
    i = x.searchsorted(lower, side="left" if inclusive else "right")
    j = x.searchsorted(upper, side="right" if inclusive else "left")
    return i, j

def between_fast(x, lower, upper, inclusive=True):
    """
    Equivalent to pd.Series.between() under the assumption that x is sorted.
    """
    i, j = between_indices(x, lower, upper, inclusive)
    if True:
        return x.iloc[i:j]
    else:
        # Mask creation is slow.
        mask = np.zeros_like(x, dtype=bool)
        mask[i:j] = True
        mask = pd.Series(mask, index=x.index)
        return x[mask]

def between(x, lower, upper, inclusive=True):
    mask = x.between(lower, upper, inclusive=inclusive)
    return x[mask]

def between_expr(x, lower, upper, inclusive=True):
    if inclusive:
        mask = (x>=lower) & (x<=upper)
    else:
        mask = (x>lower) & (x<upper)
    return x[mask]

def benchmark(func, x, lowers, uppers):
    for l,u in zip(lowers, uppers):
        func(x,lower=l,upper=u)

n_samples = 1000
n_reps = 100
x = pd.Series(np.random.randn(n_samples))
# Sort the Series.
# For pandas>=1.0:
# x = x.sort_values(ignore_index=True)
x = x.sort_values().reset_index(drop=True)

# Assert equivalence of different methods.
assert(between_fast(x, 0, 1, True ).equals(between(x, 0, 1, True)))
assert(between_expr(x, 0, 1, True ).equals(between(x, 0, 1, True)))
assert(between_fast(x, 0, 1, False).equals(between(x, 0, 1, False)))
assert(between_expr(x, 0, 1, False).equals(between(x, 0, 1, False)))

# Benchmark repeated evaluations of between().
uppers = np.linspace(0, 3, n_reps)
lowers = -uppers
%timeit benchmark(between_fast, x, lowers, uppers)
%timeit benchmark(between, x, lowers, uppers)
%timeit benchmark(between_expr, x, lowers, uppers)