Python 我一直在尝试应用fuzzyfuzzy包来解决查找欺诈条目的问题。如何在下面的问题中应用相同的方法?
我想在下表中使用fuzzyfuzzy包Python 我一直在尝试应用fuzzyfuzzy包来解决查找欺诈条目的问题。如何在下面的问题中应用相同的方法?,python,python-3.x,pandas,nlp,fuzzy-logic,Python,Python 3.x,Pandas,Nlp,Fuzzy Logic,我想在下表中使用fuzzyfuzzy包 x Reference amount 121 TOR1234 500 121 T0R1234 500 121 W7QWER 500 121 W1QWER 500 141 TRYCATC 700 141 TRYCATC 700 151 I678MKV 300 151 1678MKV 300 我想对列“x”和“amount”匹配的表进行
x Reference amount
121 TOR1234 500
121 T0R1234 500
121 W7QWER 500
121 W1QWER 500
141 TRYCATC 700
141 TRYCATC 700
151 I678MKV 300
151 1678MKV 300
这是为了检测欺诈条目,如表中的“1”替换为“I”,而“0”替换为“O”。如果您有其他解决方案,请提出。据我所知,您不需要
fuzzyfuzzy
包方法
将simple与keep=False一起使用
df = pd.DataFrame(data={"x":[121,121,121,121,141,141,151,151],
"Refrence":["TOR1234","T0R1234","W7QWER","W1QWER","TRYCATC","TRYCATC"
,"I678MKV","1678MKV"],
"amount":[500,500,500,500,700,700,300,300]})
res = df.drop_duplicates(['x','Refrence','amount'],keep=False).sort_values(['x'],ascending=[False])
print(res)
将levenshtein距离应用于同一x范围内的参照
这是为了检测欺诈条目,如表中的“1”替换为“I”,而“0”替换为“O”。如果你有其他的解决方案,请提出建议。因此,有一件事levenshtein距离
在删除重复项后,您可以在同一个x范围内计算引用的levenshtein距离,这将建议您在引用之间进行多少次编辑,使其等于库中的一个,此外,还有许多事情需要研究levenshtein距离-我尝试了这项工作解决了这个问题,但未能成功。如果你能帮我实施,那就太好了great@ROHITMUNDHRA我已经应用了levenshtein distance
并找出了两个引用中哪些字符不常见,现在请尝试如何替换引用中的那些字符串
df = pd.DataFrame(data={"x":[121,121,121,121,141,141,151,151],
"Refrence":["TOR1234","T0R1234","W7QWER","W1QWER","TRYCATC","TRYCATC"
,"I678MKV","1678MKV"],
"amount":[500,500,500,500,700,700,300,300]})
res = df.drop_duplicates(['x','Refrence','amount'],keep=False).sort_values(['x'],ascending=[False])
print(res)
x Refrence amount
6 151 I678MKV 300
7 151 1678MKV 300
0 121 TOR1234 500
1 121 T0R1234 500
2 121 W7QWER 500
3 121 W1QWER 500
from itertools import combinations
from similarity.damerau import Damerau
levenshtien = Damerau()
data = list(combinations(res['Refrence'], 2))
refrence_df = pd.DataFrame(data,columns=['Refrence','Refrence2'])
refrence_df = pd.merge(refrence_df,df[['x','Refrence']],on=['Refrence'],how='left')
refrence_df = pd.merge(refrence_df,df[['x','Refrence']],left_on=['Refrence2'],right_on=['Refrence'],how='left')
refrence_df.rename(columns={'x_x':'x_1','x_y':'x_2','Refrence_x':'Refrence'},inplace=True)
refrence_df.drop(['Refrence_y'],axis=1,inplace=True)
refrence_df = refrence_df[refrence_df['x_1']==refrence_df['x_2']]
refrence_df['edit_required'] = refrence_df.apply(lambda x: levenshtien.distance(x['Refrence'],x['Refrence2']),
axis=1)
refrence_df['characters_not_common'] = refrence_df.apply(lambda x :list(set(x['Refrence'])-set(x['Refrence2'])),axis=1)
print(refrence_df)
Refrence Refrence2 x_1 x_2 edit_required characters_not_common
0 I678MKV 1678MKV 151 151 1 [I]
9 TOR1234 T0R1234 121 121 1 [O]
10 TOR1234 W7QWER 121 121 7 [O, T, 1, 3, 2, 4]
11 TOR1234 W1QWER 121 121 7 [O, T, 3, 2, 4]
12 T0R1234 W7QWER 121 121 7 [T, 1, 0, 3, 2, 4]
13 T0R1234 W1QWER 121 121 7 [T, 0, 3, 2, 4]
14 W7QWER W1QWER 121 121 1 [7]