如何使用python/numpy计算百分位数?
是否有一种方便的方法来计算序列或一维numpy数组的百分位数 我正在寻找类似Excel的百分位函数如何使用python/numpy计算百分位数?,python,numpy,statistics,numpy-ndarray,percentile,Python,Numpy,Statistics,Numpy Ndarray,Percentile,是否有一种方便的方法来计算序列或一维numpy数组的百分位数 我正在寻找类似Excel的百分位函数 def percentile(N, P): """ Find the percentile of a list of values @parameter N - A list of values. N must be sorted. @parameter P - A float value from 0.0 to 1.0 @return - The p
def percentile(N, P):
"""
Find the percentile of a list of values
@parameter N - A list of values. N must be sorted.
@parameter P - A float value from 0.0 to 1.0
@return - The percentile of the values.
"""
n = int(round(P * len(N) + 0.5))
return N[n-1]
# A = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
# B = (15, 20, 35, 40, 50)
#
# print percentile(A, P=0.3)
# 4
# print percentile(A, P=0.8)
# 9
# print percentile(B, P=0.3)
# 20
# print percentile(B, P=0.8)
# 50
我查阅了NumPy的统计资料,但没有找到这个。我所能找到的只是中位数(第50个百分位数),但不是更具体的数字。您可能对该软件包感兴趣。它有你想要的和许多其他统计上的好东西
numpy
中的percentile()
import numpy as np
a = np.array([1,2,3,4,5])
p = np.percentile(a, 50) # return 50th percentile, e.g median.
print p
3.0
让我相信他们不会很快将percentile()
集成到numpy中。顺便说一句,如果你不想依赖scipy的话。该功能复制如下:
## {{{ http://code.activestate.com/recipes/511478/ (r1)
import math
import functools
def percentile(N, percent, key=lambda x:x):
"""
Find the percentile of a list of values.
@parameter N - is a list of values. Note N MUST BE already sorted.
@parameter percent - a float value from 0.0 to 1.0.
@parameter key - optional key function to compute value from each element of N.
@return - the percentile of the values
"""
if not N:
return None
k = (len(N)-1) * percent
f = math.floor(k)
c = math.ceil(k)
if f == c:
return key(N[int(k)])
d0 = key(N[int(f)]) * (c-k)
d1 = key(N[int(c)]) * (k-f)
return d0+d1
# median is 50th percentile.
median = functools.partial(percentile, percent=0.5)
## end of http://code.activestate.com/recipes/511478/ }}}
检查scipy.stats模块:
scipy.stats.scoreatpercentile
我通常看到的百分位的定义期望作为一个结果,从提供的列表中找到p%的值。。。这意味着结果必须来自集合,而不是集合元素之间的插值。要实现这一点,可以使用更简单的函数
def percentile(N, P):
"""
Find the percentile of a list of values
@parameter N - A list of values. N must be sorted.
@parameter P - A float value from 0.0 to 1.0
@return - The percentile of the values.
"""
n = int(round(P * len(N) + 0.5))
return N[n-1]
# A = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
# B = (15, 20, 35, 40, 50)
#
# print percentile(A, P=0.3)
# 4
# print percentile(A, P=0.8)
# 9
# print percentile(B, P=0.3)
# 20
# print percentile(B, P=0.8)
# 50
如果您希望从提供的列表中获取p%或以下的值,请使用以下简单修改:
def percentile(N, P):
n = int(round(P * len(N) + 0.5))
if n > 1:
return N[n-2]
else:
return N[0]
或者使用@ijustlovemath建议的简化:
def percentile(N, P):
n = max(int(round(P * len(N) + 0.5)), 2)
return N[n-2]
这里介绍了如何在不使用numpy的情况下实现这一点,只使用python来计算百分比
import math
def percentile(data, perc: int):
size = len(data)
return sorted(data)[int(math.ceil((size * perc) / 100)) - 1]
percentile([10.0, 9.0, 8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0], 90)
# 9.0
percentile([142, 232, 290, 120, 274, 123, 146, 113, 272, 119, 124, 277, 207], 50)
# 146
要计算序列的百分位数,请运行:
from scipy.stats import rankdata
import numpy as np
def calc_percentile(a, method='min'):
if isinstance(a, list):
a = np.asarray(a)
return rankdata(a, method=method) / float(len(a))
例如:
a = range(20)
print {val: round(percentile, 3) for val, percentile in zip(a, calc_percentile(a))}
>>> {0: 0.05, 1: 0.1, 2: 0.15, 3: 0.2, 4: 0.25, 5: 0.3, 6: 0.35, 7: 0.4, 8: 0.45, 9: 0.5, 10: 0.55, 11: 0.6, 12: 0.65, 13: 0.7, 14: 0.75, 15: 0.8, 16: 0.85, 17: 0.9, 18: 0.95, 19: 1.0}
如果您需要答案作为输入numpy数组的成员:
加上numpy中的百分位函数在默认情况下将输出计算为输入向量中两个相邻项的线性加权平均值。在某些情况下,人们可能希望返回的百分位是向量的实际元素,在这种情况下,从v1.9.0开始,您可以使用“插值”选项,选择“较低”、“较高”或“最近”
后者是向量中的一个实际条目,而前者是两个向量条目的线性插值,这两个向量条目围绕一系列:用于描述函数的百分位
假设您的df包含以下列sales和id。您想计算sales的百分位数,那么它的工作原理如下
df['sales'].describe(percentiles = [0.0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1])
0.0: .0: minimum
1: maximum
0.1 : 10th percentile and so on
从Python 3.8开始,标准库附带了作为模块一部分的函数:
返回给定分布dist
的n-1
切点列表,这些切点分隔n
分位数间隔(将dist
划分为n
连续间隔,概率相等):
统计学.分位数(dist,*,n=4,method='exclusive')
其中,n
,在我们的例子中(百分位数
)是100
计算一维numpy序列或矩阵的百分位数的便捷方法是使用numpy.percentile。例如:
import numpy as np
a = np.array([0,1,2,3,4,5,6,7,8,9,10])
p50 = np.percentile(a, 50) # return 50th percentile, e.g median.
p90 = np.percentile(a, 90) # return 90th percentile.
print('median = ',p50,' and p90 = ',p90) # median = 5.0 and p90 = 9.0
但是,如果数据中存在任何NaN值,则上述函数将不会有用。在这种情况下建议使用的函数是numpy.nanpercentile函数:
import numpy as np
a_NaN = np.array([0.,1.,2.,3.,4.,5.,6.,7.,8.,9.,10.])
a_NaN[0] = np.nan
print('a_NaN',a_NaN)
p50 = np.nanpercentile(a_NaN, 50) # return 50th percentile, e.g median.
p90 = np.nanpercentile(a_NaN, 90) # return 90th percentile.
print('median = ',p50,' and p90 = ',p90) # median = 5.5 and p90 = 9.1
在上述两个选项中,您仍然可以选择插值模式。按照下面的例子更容易理解
import numpy as np
b = np.array([1,2,3,4,5,6,7,8,9,10])
print('percentiles using default interpolation')
p10 = np.percentile(b, 10) # return 10th percentile.
p50 = np.percentile(b, 50) # return 50th percentile, e.g median.
p90 = np.percentile(b, 90) # return 90th percentile.
print('p10 = ',p10,', median = ',p50,' and p90 = ',p90)
#p10 = 1.9 , median = 5.5 and p90 = 9.1
print('percentiles using interpolation = ', "linear")
p10 = np.percentile(b, 10,interpolation='linear') # return 10th percentile.
p50 = np.percentile(b, 50,interpolation='linear') # return 50th percentile, e.g median.
p90 = np.percentile(b, 90,interpolation='linear') # return 90th percentile.
print('p10 = ',p10,', median = ',p50,' and p90 = ',p90)
#p10 = 1.9 , median = 5.5 and p90 = 9.1
print('percentiles using interpolation = ', "lower")
p10 = np.percentile(b, 10,interpolation='lower') # return 10th percentile.
p50 = np.percentile(b, 50,interpolation='lower') # return 50th percentile, e.g median.
p90 = np.percentile(b, 90,interpolation='lower') # return 90th percentile.
print('p10 = ',p10,', median = ',p50,' and p90 = ',p90)
#p10 = 1 , median = 5 and p90 = 9
print('percentiles using interpolation = ', "higher")
p10 = np.percentile(b, 10,interpolation='higher') # return 10th percentile.
p50 = np.percentile(b, 50,interpolation='higher') # return 50th percentile, e.g median.
p90 = np.percentile(b, 90,interpolation='higher') # return 90th percentile.
print('p10 = ',p10,', median = ',p50,' and p90 = ',p90)
#p10 = 2 , median = 6 and p90 = 10
print('percentiles using interpolation = ', "midpoint")
p10 = np.percentile(b, 10,interpolation='midpoint') # return 10th percentile.
p50 = np.percentile(b, 50,interpolation='midpoint') # return 50th percentile, e.g median.
p90 = np.percentile(b, 90,interpolation='midpoint') # return 90th percentile.
print('p10 = ',p10,', median = ',p50,' and p90 = ',p90)
#p10 = 1.5 , median = 5.5 and p90 = 9.5
print('percentiles using interpolation = ', "nearest")
p10 = np.percentile(b, 10,interpolation='nearest') # return 10th percentile.
p50 = np.percentile(b, 50,interpolation='nearest') # return 50th percentile, e.g median.
p90 = np.percentile(b, 90,interpolation='nearest') # return 90th percentile.
print('p10 = ',p10,', median = ',p50,' and p90 = ',p90)
#p10 = 2 , median = 5 and p90 = 9
如果您的输入数组只包含整数值,您可能会对作为整数的百分比答案感兴趣。如果是这样,请选择插值模式,如“较低”、“较高”或“最近”。我引导数据,然后绘制10个样本的置信区间。置信区间表示概率介于5%和95%之间的范围
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
import json
import dc_stat_think as dcst
data = [154, 400, 1124, 82, 94, 108]
#print (np.percentile(data,[0.5,95])) # gives the 95th percentile
bs_data = dcst.draw_bs_reps(data, np.mean, size=6*10)
#print(np.reshape(bs_data,(24,6)))
x= np.linspace(1,6,6)
print(x)
for (item1,item2,item3,item4,item5,item6) in bs_data.reshape((10,6)):
line_data=[item1,item2,item3,item4,item5,item6]
ci=np.percentile(line_data,[.025,.975])
mean_avg=np.mean(line_data)
fig, ax = plt.subplots()
ax.plot(x,line_data)
ax.fill_between(x, (line_data-ci[0]), (line_data+ci[1]), color='b', alpha=.1)
ax.axhline(mean_avg,color='red')
plt.show()
非常感谢。那就是它一直藏身的地方。我知道scipy,但我想我假设像percentiles这样的简单东西会被内置到numpy中。到目前为止,numpy中存在一个百分位函数:您也可以将其用作聚合函数,例如,要按键计算值列每组的第十个百分位,请使用df.groupby('key')[['value']]].agg(lambda g:np.percentile(g,10))
请注意,SciPy建议对NumPy 1.9和更高版本使用np.percentile。我是上述配方的作者。ASPN中的一位评论员指出原始代码有一个bug。公式应为d0=键(N[int(f)])*(c-k);d1=键(N[int(c)])*(k-f)。它已在ASPN上更正。百分位数
如何知道对N
使用什么?它没有在函数调用中指定。对于那些甚至没有阅读代码的人,在使用它之前,必须对N进行排序。我对lambda表达式感到困惑。它做什么?它是如何做的?我知道lambda表达式是什么,所以我不是在问lambda是什么。我在问这个特定的lambda表达式是做什么的,它是如何一步一步地做的?谢谢lambda函数允许您在计算百分比之前转换N
中的数据。假设您实际有一个元组列表N=[(1,2),(3,1),…,(5,1)]
并且您想要得到元组第一个元素的百分位,然后选择key=lambda x:x[0]
。在计算百分位之前,您还可以对列表元素应用一些(顺序更改)转换。谢谢,我还希望百分位/中位数从集合中得到实际值,而不是插值hi@mpouncet。谢谢你提供了上面的代码。为什么百分位数总是返回整数值?percentile函数应该返回值列表的第N个百分位数,这也可以是一个浮点数。例如,ExcelPERCENTILE
函数为上面的示例返回以下百分位:3.7=百分位(A,P=0.3)
,0.82=百分位(A,P=0.8)
,20=百分位(B,P=0.3)
,42=百分位(B,P=0.8)
。第一句解释了这一点。百分位数更常见的定义是,它是一个系列中的数字,低于该系列中P%的值。因为这是列表中项目的索引号,所以
import numpy as np
b = np.array([1,2,3,4,5,6,7,8,9,10])
print('percentiles using default interpolation')
p10 = np.percentile(b, 10) # return 10th percentile.
p50 = np.percentile(b, 50) # return 50th percentile, e.g median.
p90 = np.percentile(b, 90) # return 90th percentile.
print('p10 = ',p10,', median = ',p50,' and p90 = ',p90)
#p10 = 1.9 , median = 5.5 and p90 = 9.1
print('percentiles using interpolation = ', "linear")
p10 = np.percentile(b, 10,interpolation='linear') # return 10th percentile.
p50 = np.percentile(b, 50,interpolation='linear') # return 50th percentile, e.g median.
p90 = np.percentile(b, 90,interpolation='linear') # return 90th percentile.
print('p10 = ',p10,', median = ',p50,' and p90 = ',p90)
#p10 = 1.9 , median = 5.5 and p90 = 9.1
print('percentiles using interpolation = ', "lower")
p10 = np.percentile(b, 10,interpolation='lower') # return 10th percentile.
p50 = np.percentile(b, 50,interpolation='lower') # return 50th percentile, e.g median.
p90 = np.percentile(b, 90,interpolation='lower') # return 90th percentile.
print('p10 = ',p10,', median = ',p50,' and p90 = ',p90)
#p10 = 1 , median = 5 and p90 = 9
print('percentiles using interpolation = ', "higher")
p10 = np.percentile(b, 10,interpolation='higher') # return 10th percentile.
p50 = np.percentile(b, 50,interpolation='higher') # return 50th percentile, e.g median.
p90 = np.percentile(b, 90,interpolation='higher') # return 90th percentile.
print('p10 = ',p10,', median = ',p50,' and p90 = ',p90)
#p10 = 2 , median = 6 and p90 = 10
print('percentiles using interpolation = ', "midpoint")
p10 = np.percentile(b, 10,interpolation='midpoint') # return 10th percentile.
p50 = np.percentile(b, 50,interpolation='midpoint') # return 50th percentile, e.g median.
p90 = np.percentile(b, 90,interpolation='midpoint') # return 90th percentile.
print('p10 = ',p10,', median = ',p50,' and p90 = ',p90)
#p10 = 1.5 , median = 5.5 and p90 = 9.5
print('percentiles using interpolation = ', "nearest")
p10 = np.percentile(b, 10,interpolation='nearest') # return 10th percentile.
p50 = np.percentile(b, 50,interpolation='nearest') # return 50th percentile, e.g median.
p90 = np.percentile(b, 90,interpolation='nearest') # return 90th percentile.
print('p10 = ',p10,', median = ',p50,' and p90 = ',p90)
#p10 = 2 , median = 5 and p90 = 9
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
import json
import dc_stat_think as dcst
data = [154, 400, 1124, 82, 94, 108]
#print (np.percentile(data,[0.5,95])) # gives the 95th percentile
bs_data = dcst.draw_bs_reps(data, np.mean, size=6*10)
#print(np.reshape(bs_data,(24,6)))
x= np.linspace(1,6,6)
print(x)
for (item1,item2,item3,item4,item5,item6) in bs_data.reshape((10,6)):
line_data=[item1,item2,item3,item4,item5,item6]
ci=np.percentile(line_data,[.025,.975])
mean_avg=np.mean(line_data)
fig, ax = plt.subplots()
ax.plot(x,line_data)
ax.fill_between(x, (line_data-ci[0]), (line_data+ci[1]), color='b', alpha=.1)
ax.axhline(mean_avg,color='red')
plt.show()