用python组合列
我有一个带有数值的数据框列:用python组合列,python,pandas,numpy,dataframe,binning,Python,Pandas,Numpy,Dataframe,Binning,我有一个带有数值的数据框列: df['percentage'].head() 46.5 44.2 100.0 42.12 我希望将列视为bin计数: bins = [0, 1, 5, 10, 25, 50, 100] 如何将结果作为带有值计数的存储箱获取 [0, 1] bin amount [1, 5] etc [5, 10] etc ...... 您可以使用: 或: …然后或合计: 默认情况下,cutreturncategorical Series方法,如Series.valu
df['percentage'].head()
46.5
44.2
100.0
42.12
我希望将列视为bin计数:
bins = [0, 1, 5, 10, 25, 50, 100]
如何将结果作为带有值计数的存储箱获取
[0, 1] bin amount
[1, 5] etc
[5, 10] etc
......
您可以使用:
或:
…然后或合计:
默认情况下,cut
returncategorical
Series
方法,如Series.value\u counts()
将使用所有类别,即使数据中不存在某些类别。使用模块进行加速。
在大数据集(500k>
)上,pd.cut
对于装箱数据来说可能非常慢
我在numba
中使用即时编译编写了自己的函数,这大约比6x
更快:
from numba import njit
@njit
def cut(arr):
bins = np.empty(arr.shape[0])
for idx, x in enumerate(arr):
if (x >= 0) & (x < 1):
bins[idx] = 1
elif (x >= 1) & (x < 5):
bins[idx] = 2
elif (x >= 5) & (x < 10):
bins[idx] = 3
elif (x >= 10) & (x < 25):
bins[idx] = 4
elif (x >= 25) & (x < 50):
bins[idx] = 5
elif (x >= 50) & (x < 100):
bins[idx] = 6
else:
bins[idx] = 7
return bins
可选:还可以将其作为字符串映射到存储箱:
a = cut(df['percentage'].to_numpy())
conversion_dict = {1: 'bin1',
2: 'bin2',
3: 'bin3',
4: 'bin4',
5: 'bin5',
6: 'bin6',
7: 'bin7'}
bins = list(map(conversion_dict.get, a))
# ['bin5', 'bin5', 'bin7', 'bin5']
速度比较:
# create dataframe of 8 million rows for testing
dfbig = pd.concat([df]*2000000, ignore_index=True)
dfbig.shape
# (8000000, 1)
如果没有垃圾箱=[0,1,5,10,25,50,100]
,我能说创建5个垃圾箱,它将按平均切割量进行切割吗?例如,我有110条记录,我想把它们分成5个箱子,每个箱子里有22条记录。@qqwww-不确定是否理解,你认为qcut
@qqwww要做到这一点,pd.cut页面中的示例显示:pd.cut(np.array([1,7,5,4,6,3]),3)将数组分成3等份。@AyanMitra-你认为df.groupby(pd.cut(df['percentage',bin=bin]).mean()
?谢谢你这个答案帮助了我:)
s = pd.cut(df['percentage'], bins=bins).value_counts()
print (s)
(25, 50] 3
(50, 100] 1
(10, 25] 0
(5, 10] 0
(1, 5] 0
(0, 1] 0
Name: percentage, dtype: int64
s = df.groupby(pd.cut(df['percentage'], bins=bins)).size()
print (s)
percentage
(0, 1] 0
(1, 5] 0
(5, 10] 0
(10, 25] 0
(25, 50] 3
(50, 100] 1
dtype: int64
from numba import njit
@njit
def cut(arr):
bins = np.empty(arr.shape[0])
for idx, x in enumerate(arr):
if (x >= 0) & (x < 1):
bins[idx] = 1
elif (x >= 1) & (x < 5):
bins[idx] = 2
elif (x >= 5) & (x < 10):
bins[idx] = 3
elif (x >= 10) & (x < 25):
bins[idx] = 4
elif (x >= 25) & (x < 50):
bins[idx] = 5
elif (x >= 50) & (x < 100):
bins[idx] = 6
else:
bins[idx] = 7
return bins
cut(df['percentage'].to_numpy())
# array([5., 5., 7., 5.])
a = cut(df['percentage'].to_numpy())
conversion_dict = {1: 'bin1',
2: 'bin2',
3: 'bin3',
4: 'bin4',
5: 'bin5',
6: 'bin6',
7: 'bin7'}
bins = list(map(conversion_dict.get, a))
# ['bin5', 'bin5', 'bin7', 'bin5']
# create dataframe of 8 million rows for testing
dfbig = pd.concat([df]*2000000, ignore_index=True)
dfbig.shape
# (8000000, 1)
%%timeit
cut(dfbig['percentage'].to_numpy())
# 38 ms ± 616 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
%%timeit
bins = [0, 1, 5, 10, 25, 50, 100]
labels = [1,2,3,4,5,6]
pd.cut(dfbig['percentage'], bins=bins, labels=labels)
# 215 ms ± 9.76 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)