Python 使用scipy对数据帧内的组进行方差分析
我有一个如下的数据帧。我需要在这三个条件之间做方差分析。数据帧看起来像:Python 使用scipy对数据帧内的组进行方差分析,python,pandas,scipy,Python,Pandas,Scipy,我有一个如下的数据帧。我需要在这三个条件之间做方差分析。数据帧看起来像: data0 = pd.DataFrame({'Names': ['CTA15', 'CTA15', 'AC007', 'AC007', 'AC007','AC007'], 'value': [22, 22, 2, 2, 2,5], 'condition':['NON', 'NON', 'YES', 'YES', 'RE','RE']}) 我需要在是与否、非与RE、是与RE之间进行方差分析测试,条件来自
data0 = pd.DataFrame({'Names': ['CTA15', 'CTA15', 'AC007', 'AC007', 'AC007','AC007'],
'value': [22, 22, 2, 2, 2,5],
'condition':['NON', 'NON', 'YES', 'YES', 'RE','RE']})
我需要在是与否、非与RE、是与RE之间进行方差分析测试,条件来自名称的条件。
我知道我可以这样做
NON=df.query('condition =="NON"and Names=="CTA15"')
no=df.value
YES=df.query('condition =="YES"and Names=="CTA15"')
Y=YES.value
然后进行单因素方差分析,如下所示:
from scipy import stats
f_val, p_val = stats.f_oneway(no, Y)
print ("One-way ANOVA P =", p_val )
但如果有任何优雅的解决方案,那就太好了,因为我的初始数据帧很大,并且有许多名称和条件可供比较请考虑以下示例数据帧:
df = pd.DataFrame({'Names': np.random.randint(1, 10, 1000),
'value': np.random.randn(1000),
'condition': np.random.choice(['NON', 'YES', 'RE'], 1000)})
df.head()
Out:
Names condition value
0 4 RE 0.844120
1 4 NON -0.440285
2 5 YES 0.559497
3 4 RE 0.472425
4 9 YES 0.205906
以下按名称对数据帧进行分组,然后将每个条件组传递给ANOVA:
import scipy.stats as ss
for name_group in df.groupby('Names'):
samples = [condition[1] for condition in name_group[1].groupby('condition')['value']]
f_val, p_val = ss.f_oneway(*samples)
print('Name: {}, F value: {:.3f}, p value: {:.3f}'.format(name_group[0], f_val, p_val))
Name: 1, F value: 0.138, p value: 0.871
Name: 2, F value: 1.458, p value: 0.237
Name: 3, F value: 0.742, p value: 0.479
Name: 4, F value: 2.718, p value: 0.071
Name: 5, F value: 0.255, p value: 0.776
Name: 6, F value: 1.731, p value: 0.182
Name: 7, F value: 0.269, p value: 0.764
Name: 8, F value: 0.474, p value: 0.624
Name: 9, F value: 1.226, p value: 0.297
对于事后测试,您可以使用statsmodels(如上所述):
名称1平均值的多重比较-Tukey HSD,FWER=0.05
============================================
第1组第2组平均上下不合格品
--------------------------------------------
非RE 0.0086-0.5129 0.5301假
非是0.0084-0.4817 0.4986假
RE是-0.0002-0.5217 0.5214假
--------------------------------------------
名称2平均值的多重比较-Tukey HSD,FWER=0.05
============================================
第1组第2组平均上下不合格品
--------------------------------------------
非RE-0.0089-0.5299 0.5121假
非是0.083-0.4182 0.5842假
RE是0.0919-0.4008 0.5846假
--------------------------------------------
名称3平均值的多重比较-Tukey HSD,FWER=0.05
============================================
第1组第2组平均上下不合格品
--------------------------------------------
非RE 0.2401-0.3136 0.7938假
非是0.2765-0.2903 0.8432假
RE是0.0364-0.5052 0.578假
--------------------------------------------
名称4平均值的多重比较-Tukey HSD,FWER=0.05
============================================
第1组第2组平均上下不合格品
--------------------------------------------
非RE 0.0894-0.5825 0.7613假
非是-0.0437-0.7418 0.6544假
RE是-0.1331-0.6949 0.4287假
--------------------------------------------
名称5平均值的多重比较-Tukey HSD,FWER=0.05
============================================
第1组第2组平均上下不合格品
--------------------------------------------
非RE-0.4264-0.9495 0.0967假
非是0.0439-0.4264 0.5142假
RE是0.4703-0.0155 0.9561假
--------------------------------------------
名称6平均值的多重比较-Tukey HSD,FWER=0.05
============================================
第1组第2组平均上下不合格品
--------------------------------------------
非RE 0.0649-0.4971 0.627假
非是-0.406-0.9405 0.1285假
RE是-0.4709-1.0136 0.0717假
--------------------------------------------
名称7平均值的多重比较-Tukey HSD,FWER=0.05
============================================
第1组第2组平均上下不合格品
--------------------------------------------
非RE 0.3111-0.2766 0.8988假
非是-0.1664-0.7314 0.3987假
RE是-0.4774-1.0688 0.114假
--------------------------------------------
名称8平均值的多重比较-Tukey HSD,FWER=0.05
============================================
第1组第2组平均上下不合格品
--------------------------------------------
非RE-0.0224-0.668 0.6233假
非是0.0119-0.668 0.6918假
RE是0.0343-0.6057 0.6742假
--------------------------------------------
名称9平均值的多重比较-Tukey HSD,FWER=0.05
============================================
第1组第2组平均上下不合格品
--------------------------------------------
非RE-0.2414-0.7792 0.2963假
非是0.0696-0.5746 0.7138假
RE是0.311-0.3129 0.935假
from statsmodels.stats.multicomp import pairwise_tukeyhsd
for name, grouped_df in df.groupby('Names'):
print('Name {}'.format(name), pairwise_tukeyhsd(grouped_df['value'], grouped_df['condition']))
Name 1 Multiple Comparison of Means - Tukey HSD,FWER=0.05
============================================
group1 group2 meandiff lower upper reject
--------------------------------------------
NON RE 0.0086 -0.5129 0.5301 False
NON YES 0.0084 -0.4817 0.4986 False
RE YES -0.0002 -0.5217 0.5214 False
--------------------------------------------
Name 2 Multiple Comparison of Means - Tukey HSD,FWER=0.05
============================================
group1 group2 meandiff lower upper reject
--------------------------------------------
NON RE -0.0089 -0.5299 0.5121 False
NON YES 0.083 -0.4182 0.5842 False
RE YES 0.0919 -0.4008 0.5846 False
--------------------------------------------
Name 3 Multiple Comparison of Means - Tukey HSD,FWER=0.05
============================================
group1 group2 meandiff lower upper reject
--------------------------------------------
NON RE 0.2401 -0.3136 0.7938 False
NON YES 0.2765 -0.2903 0.8432 False
RE YES 0.0364 -0.5052 0.578 False
--------------------------------------------
Name 4 Multiple Comparison of Means - Tukey HSD,FWER=0.05
============================================
group1 group2 meandiff lower upper reject
--------------------------------------------
NON RE 0.0894 -0.5825 0.7613 False
NON YES -0.0437 -0.7418 0.6544 False
RE YES -0.1331 -0.6949 0.4287 False
--------------------------------------------
Name 5 Multiple Comparison of Means - Tukey HSD,FWER=0.05
============================================
group1 group2 meandiff lower upper reject
--------------------------------------------
NON RE -0.4264 -0.9495 0.0967 False
NON YES 0.0439 -0.4264 0.5142 False
RE YES 0.4703 -0.0155 0.9561 False
--------------------------------------------
Name 6 Multiple Comparison of Means - Tukey HSD,FWER=0.05
============================================
group1 group2 meandiff lower upper reject
--------------------------------------------
NON RE 0.0649 -0.4971 0.627 False
NON YES -0.406 -0.9405 0.1285 False
RE YES -0.4709 -1.0136 0.0717 False
--------------------------------------------
Name 7 Multiple Comparison of Means - Tukey HSD,FWER=0.05
============================================
group1 group2 meandiff lower upper reject
--------------------------------------------
NON RE 0.3111 -0.2766 0.8988 False
NON YES -0.1664 -0.7314 0.3987 False
RE YES -0.4774 -1.0688 0.114 False
--------------------------------------------
Name 8 Multiple Comparison of Means - Tukey HSD,FWER=0.05
============================================
group1 group2 meandiff lower upper reject
--------------------------------------------
NON RE -0.0224 -0.668 0.6233 False
NON YES 0.0119 -0.668 0.6918 False
RE YES 0.0343 -0.6057 0.6742 False
--------------------------------------------
Name 9 Multiple Comparison of Means - Tukey HSD,FWER=0.05
============================================
group1 group2 meandiff lower upper reject
--------------------------------------------
NON RE -0.2414 -0.7792 0.2963 False
NON YES 0.0696 -0.5746 0.7138 False
RE YES 0.311 -0.3129 0.935 False