Pandas 设置的并集不正确

Pandas 设置的并集不正确,pandas,dataframe,union,Pandas,Dataframe,Union,我写这些文件只是为了检查结果。但是我的集合是不正确的。我需要获得诊断过滤器的联合=[FilterDiagnosism,FilterDiagnosism,FilterDiagnosism,FilterDiagnosism,FilterDiagnosism] 当前正在返回另外两行 843786米、12.45米、15.7米、82.57米、477.1米 869691米11.8米16.58米78.99米432米 原始文件如下所示 df = pd.DataFrame({'id': ['842302', '8

我写这些文件只是为了检查结果。但是我的集合是不正确的。我需要获得诊断过滤器的联合=[FilterDiagnosism,FilterDiagnosism,FilterDiagnosism,FilterDiagnosism,FilterDiagnosism] 当前正在返回另外两行

843786米、12.45米、15.7米、82.57米、477.1米

869691米11.8米16.58米78.99米432米

原始文件如下所示

df = pd.DataFrame({'id': ['842302', '842303', '842304'], 
               'diagnosis': ['M', 'B', 'M'],
               'radius_mean': [20.57, 17.3, 13.3],
               'perimeter_mean': [20.57, 20.57, 20.57],
               'area_mean': [206.57, 206.57, 240.57],})

下面是我设置的工会代码

diagnosisFilters = [filterRDiagnosisM,filterTDiagnosisM,filterPDiagnosisM,filterADiagnosisM]
diagnosisResult = reduce(lambda  left,right: pd.merge(left,right,how='outer',on=["id"]), diagnosisFilters)
我将把它留在这里作为上下文,直到我找到一种更干净的方式来显示它,而不显示整个文件

import pandas as pd
from functools import reduce
import os

#FOLDERS
vf = './valueFilters'
df = './diagnosisFilters'
results = './results'
if not os.path.exists(vf):
    os.mkdir(vf)
if not os.path.exists(df):
    os.mkdir(df)
if not os.path.exists(results):
    os.mkdir(results)

#DATAFRAMES
my_csv = pd.read_csv('breastCancerDataReducedDimensions.csv')
radius = pd.DataFrame(my_csv[['id','diagnosis', 'radius_mean']])
texture = pd.DataFrame(my_csv[['id','diagnosis','texture_mean']])
perimeter = pd.DataFrame(my_csv[['id','diagnosis','perimeter_mean']])
area = pd.DataFrame(my_csv[['id','diagnosis','area_mean']])

#RADIUS
filterRadius = radius[radius['radius_mean'] >= 13.0]
filterRadius.to_csv(vf+'/q3_gte_13.csv',index=False)
#B
filterRDiagnosisB = radius[radius['diagnosis'] == 'B']
filterRDiagnosisB.to_csv(df+'/q3_B.csv',index=False)
#M
filterRDiagnosisM = radius[radius['diagnosis'] == 'M']
filterRDiagnosisM.to_csv(df+'/q3_M.csv',index=False)

#TEXTURE
filterTexture = texture[texture['texture_mean'] >= 18.0]
filterTexture.to_csv(vf+'/q4_gte_18.csv',index=False)
#B
filterTDiagnosisB = texture[texture['diagnosis'] == 'B']
filterTDiagnosisB.to_csv(df+'/q4_B.csv',index=False)
#M
filterTDiagnosisM = texture[texture['diagnosis'] == 'M']
filterTDiagnosisM.to_csv(df+'/q4_M.csv',index=False)

#PERIMETER
filterPerimeter = perimeter[perimeter['perimeter_mean'] >= 85.0]
filterPerimeter.to_csv(vf+'/q5_gte_85.csv',index=False)
#B
filterPDiagnosisB = perimeter[perimeter['diagnosis'] == 'B']
filterPDiagnosisB.to_csv(df+'/q5_B.csv',index=False)
#M
filterPDiagnosisM = perimeter[perimeter['diagnosis'] == 'M']
filterPDiagnosisM.to_csv(df+'/q5_M.csv',index=False)

#AREA
filterArea = area[area['area_mean'] >= 500.0]
filterArea.to_csv(vf+'/q6_gte_500.csv',index=False)
#B
filterADiagnosisB = area[area['diagnosis'] == 'B']
filterADiagnosisB.to_csv(df+'/q6_B.csv',index=False)
#M
filterADiagnosisM = area[area['diagnosis'] == 'M']
filterADiagnosisM.to_csv(df+'/q6_M.csv',index=False)

#RESULT
intValueFilters = [filterRadius,filterTexture,filterPerimeter,filterArea]
valueResult = reduce(lambda  left,right: pd.merge(left,right,how='inner',on=["id"]), intValueFilters)
valueResult.to_csv(results+'/NewResult.csv',index=False)

#CHECK RESULT - METHOD 1
diagnosisFilters = [filterRDiagnosisM,filterTDiagnosisM,filterPDiagnosisM,filterADiagnosisM]
diagnosisResult = reduce(lambda  left,right: pd.merge(left,right,how='outer',on=["id"]), diagnosisFilters)
diagnosisResult.to_csv(results+'/SubsetMResult.csv',index=False)

difference1 = diagnosisResult[~diagnosisResult.apply(tuple,1).isin(valueResult.apply(tuple,1))]
pd.DataFrame(difference1).to_csv(results+'/difference1.csv',index=False)

#CHECK RESULT - METHOD 2
originalResult = my_csv[my_csv['diagnosis'] == 'M']
originalResult.to_csv(results+'/OriginalResult.csv',index=False)

difference2 = originalResult[~originalResult.apply(tuple,1).isin(valueResult.apply(tuple,1))]
pd.DataFrame(difference2).to_csv(results+'/difference2.csv',index=False)

#COMPLETE
print('complete')

固定的。问题不在集合操作中。我过滤不正确。所以输入文件的位置不正确。我花了一点时间来解释为什么会出现这种情况,但最后我需要过滤一个过滤后的df,但我过滤了未过滤的df

filterRDiagnosisB = radius[radius['diagnosis'] == 'B']
改为:

filterRDiagnosisB = filterRadius[filterRadius['diagnosis'] == 'B']

嗨,欢迎来到SO!这里有很多代码和数据。我怀疑只有一小部分是必要的,以确定您的问题,但助手可能必须筛选无关的细节。你能把这个减到一个小数目吗?这将帮助您更快地获得更好的答案。这也可能有帮助:将整个代码放在上下文的底部。我已经把它从位置上移开了,我想现在钟摆已经向另一个方向摆得太远了。测试结果应是可复制的;目前没有足够的资源来运行您的代码。一个很好的指导原则是:提供必要的最短代码,以便有人可以复制到他们的终端并运行,这样他们就可以调试了不管怎样,我不明白问题是什么,这些行存在的事实有什么不对?现在添加回来。问题是行不应该在那里,因为它超出了筛选条件。