Python 在每个类别的分类数据中过滤异常值

我不熟悉pandas/seaborn/etc，尝试以不同的风格（使用seaborn）绘制我的数据子集，使用如下示例：

我的目标是仅绘制每个

x

/

hue

维度内的异常值，即，对于所示的示例，我将使用8个不同的百分比截止值来显示8列不同的点

我有一个数据帧，如：

        Cat  RPS   latency_ns
0       X    100     909423.0
1       X    100   14747385.0
2       X    1000  14425058.0
3       Y    100    7107907.0
4       Y    1000  21466101.0
...     ...  ...   ...

我想过滤这些数据，只留下99.9%以上的异常值

我发现我可以做到：

df.groupby([dim1_label, dim2_label]).quantile(0.999)

要获得类似于：

                                         latency_ns
Cat                          RPS                   
X                            10 RPS    6.463337e+07
                             100 RPS   4.400980e+07
                             1000 RPS  6.075070e+07
Y                            100 RPS   3.958944e+07
Z                            10 RPS    5.621427e+07
                             100 RPS   4.436208e+07
                             1000 RPS  6.658783e+07

---

但我不确定合并/筛选操作的方向。

这里是我创建的一个小示例，用于指导您。我希望这是有帮助的

代码

                                         latency_ns
Cat                          RPS                   
X                            10 RPS    6.463337e+07
                             100 RPS   4.400980e+07
                             1000 RPS  6.075070e+07
Y                            100 RPS   3.958944e+07
Z                            10 RPS    5.621427e+07
                             100 RPS   4.436208e+07
                             1000 RPS  6.658783e+07

import numpy as np
import pandas as pd
import seaborn as sns

#create a sample data frame
n = 1000
prng = np.random.RandomState(123)

x = prng.uniform(low=1, high=5, size=(n,)).astype('int')
#print(x[:10])
#[3 2 1 3 3 2 4 3 2 2]

y = prng.normal(size=(n,))
#print(y[:10])
#[ 1.32327371 -0.00315484 -0.43065984 -0.14641577  1.16017595 -0.64151234
#-0.3002324  -0.63226078 -0.20431653  0.2136956 ]

z = prng.binomial(n=1,p=2/3,size=(n,))
#print(z[:10])
#[1 0 1 1 1 1 0 1 1 1]

#analagously to the smoking example, my df x maps day,
#y maps to total bill, and z maps to is smoker (or not)
df = pd.DataFrame(data={'x':x,'y':y,'z':z})

#df.head()

df_filtered = pd.DataFrame()

#df.groupby.quantile([0.9]) returns a scalar, unless you want to plot only a single point, use this
#if you want to plot values that are within the lower and upper bounds, then some
#conditional filtering is required, see the conditional filtering I wrote below
for i,j in df.groupby([x, z]):
    b = j.quantile([0,0.9]) #use [0.999,1] in your case
    lb = b['y'].iloc[0]
    ub = b['y'].iloc[1]
    df_temp = j[(j['y']>=lb)&(j['y']<=ub)]
    df_filtered = pd.concat([df_filtered,df_temp])


#print(df_filtered.count())
#x    897
#y    897
#z    897
#dtype: int64

import matplotlib.pyplot as plt

ax = sns.stripplot(x='x', y='y', hue='z',
data=df_filtered, jitter=True,
palette="Set2", dodge=True)

plt.show()