Python用于groupby的求和操作,但不包括非数字数据
如何使用python中csv文件中的groupby执行求和操作,但从groupby中排除一些非数字数据?例如,我有csv文件:Python用于groupby的求和操作,但不包括非数字数据,python,pandas,csv,dataframe,sum,Python,Pandas,Csv,Dataframe,Sum,如何使用python中csv文件中的groupby执行求和操作,但从groupby中排除一些非数字数据?例如,我有csv文件: id | filename | #Line_Changed ----------------------------------------------- 1 | analyze/dir_list.txt | 16 2 | metrics/metrics1.csv |
id | filename | #Line_Changed
-----------------------------------------------
1 | analyze/dir_list.txt | 16
2 | metrics/metrics1.csv | 11
3 | metrics/metrics2.csv | 15
4 | analyze/dir_list.txt | =>
5 | metrics/metrics1.csv | 11
6 | metrics/metrics2.csv | bin
7 | metrics/metrics2.csv | 4
8 | analyze/dir_list.txt | 4
我想按列文件名分组,只计算只有数字数据的行的总和,不包括非数字数据。结果应该如下所示:
filename | SUM #Line_Changed
-----------------------------------------------
analyze/dir_list.txt | 20
metrics/metrics1.csv | 22
metrics/metrics2.csv | 19
到目前为止我所做的:
df = pd.read_csv('diffhistogram.csv')
by_fn = df.groupby('filename')
mydata = {}
for name in ['#line_changed']:
mydata['SUM ' + name] = by_fn[name].sum()
output = pd.DataFrame(mydata)
print(output)
但输出假定列“#line_changed”中的数据为字符串:
filename | SUM #Line_Changed
-----------------------------------------------
analyze/dir_list.txt | 16=>4
metrics/metrics1.csv | 1111
metrics/metrics2.csv | 15bin4
有没有办法指定要在sum()操作中包含哪些数值数据以及要排除哪些非数值数据?我认为您需要使用参数errors='concurve'
将非数值数据转换为NaN
s,然后groupby
+sum
忽略以下行:
df = (pd.to_numeric(df['#Line_Changed'], errors='coerce')
.groupby(df['filename'])
.sum()
.to_frame()
.add_prefix('SUM ')
.reset_index())
print (df)
filename SUM #Line_Changed
0 analyze/dir_list.txt 20.0
1 metrics/metrics1.csv 22.0
2 metrics/metrics2.csv 19.0
或分配给用于groupby
的新列:
df['SUM #Line_Changed'] = pd.to_numeric(df['#Line_Changed'], errors='coerce')
df = df.groupby('filename', as_index=False)['SUM #Line_Changed'].sum()
print (df)
filename SUM #Line_Changed
0 analyze/dir_list.txt 20.0
1 metrics/metrics1.csv 22.0
2 metrics/metrics2.csv 19.0
详细信息:
df['SUM #Line_Changed'] = pd.to_numeric(df['#Line_Changed'], errors='coerce')
print (df)
id filename #Line_Changed SUM #Line_Changed
0 1 analyze/dir_list.txt 16 16.0
1 2 metrics/metrics1.csv 11 11.0
2 3 metrics/metrics2.csv 15 15.0
3 4 analyze/dir_list.txt => NaN
4 5 metrics/metrics1.csv 11 11.0
5 6 metrics/metrics2.csv bin NaN
6 7 metrics/metrics2.csv 4 4.0
7 8 analyze/dir_list.txt 4 4.0
编辑:
如果要从原始数据框中删除非数字行,请执行以下操作:
df['#Line_Changed'] = pd.to_numeric(df['#Line_Changed'], errors='coerce')
df = df.dropna(subset=['#Line_Changed'])
print (df)
id filename #Line_Changed
0 1 analyze/dir_list.txt 16.0
1 2 metrics/metrics1.csv 11.0
2 3 metrics/metrics2.csv 15.0
4 5 metrics/metrics1.csv 11.0
6 7 metrics/metrics2.csv 4.0
7 8 analyze/dir_list.txt 4.0
这很有效。这很有帮助。但如何通过排除NaN数据在条形图中绘制此数据框?我试图绘制它们,但NaN数据显示在图表中。@Yusuf-Do yo uwant plot column
SUM\Line\u已更改
?然后使用df=df.dropna(子集=['SUM#Line_Changed'])