如何使用pythons内置的map和reduce函数计算字符串中的字母频率
我想使用pythons映射和reduce内置函数计算字符串中字母的频率。有人能告诉我怎么做吗 到目前为止,我得到的是:如何使用pythons内置的map和reduce函数计算字符串中的字母频率,python,map,reduce,frequency-distribution,Python,Map,Reduce,Frequency Distribution,我想使用pythons映射和reduce内置函数计算字符串中字母的频率。有人能告诉我怎么做吗 到目前为止,我得到的是: s = "the quick brown fox jumped over the lazy dog" # Map function m = lambda x: (x,1) # Reduce # Add the two frequencies if they are the same # else.... Not sure how to put both back in th
s = "the quick brown fox jumped over the lazy dog"
# Map function
m = lambda x: (x,1)
# Reduce
# Add the two frequencies if they are the same
# else.... Not sure how to put both back in the list
# in the case where they are not the same.
r = lambda x,y: (x[0], x[1] + y[1]) if x[0] == y[0] else ????
freq = reduce(r, map(m, s))
当所有字母都相同时,这非常有效
>>> s
'aaaaaaa'
>>> map(m, s)
[('a', 1), ('a', 1), ('a', 1), ('a', 1), ('a', 1), ('a', 1), ('a', 1)]
>>> reduce(r, map(m, s))
('a', 7)
当存在不同的字母时,如何使其正常工作?暂时回避有关代码的问题,我将指出,计数的常用(也是最快)方法之一是使用collections模块中的Counter类。下面是它在Python 2.7.3解释器中的使用示例:
>>> from collections import Counter
>>> lets=Counter('aaaaabadfasdfasdfafsdff')
>>> lets
Counter({'a': 9, 'f': 6, 'd': 4, 's': 3, 'b': 1})
>>> s = "the quick brown fox jumped over the lazy dog"
>>> Counter(s)
Counter({' ': 8, 'e': 4, 'o': 4, 'd': 2, 'h': 2, 'r': 2, 'u': 2, 't': 2, 'a': 1, 'c': 1, 'b': 1, 'g': 1, 'f': 1, 'i': 1, 'k': 1, 'j': 1, 'm': 1, 'l': 1, 'n': 1, 'q': 1, 'p': 1, 'w': 1, 'v': 1, 'y': 1, 'x': 1, 'z': 1})
要使用reduce,请定义一个辅助函数addto(oldtoal,newitem)
,该函数将newitem
添加到oldtoal
并返回一个新的总数。总计的初始值设定项是一个空字典,{}
。这里是一个解释过的例子。请注意,get()的第二个参数是当键尚未在字典中时使用的默认值
>>> def addto(d,x):
... d[x] = d.get(x,0) + 1
... return d
...
>>> reduce (addto, s, {})
{' ': 8, 'a': 1, 'c': 1, 'b': 1, 'e': 4, 'd': 2, 'g': 1, 'f': 1, 'i': 1, 'h': 2, 'k': 1, 'j': 1, 'm': 1, 'l': 1, 'o': 4, 'n': 1, 'q': 1, 'p': 1, 'r': 2, 'u': 2, 't': 2, 'w': 1, 'v': 1, 'y': 1, 'x': 1, 'z': 1}
下面显示的代码打印了几个方法中每个方法1000次的执行时间。在旧的AMD Athlon 5000+Linux 3.2.0-32 Ubuntu 12系统上执行时,使用两个不同的字符串s
打印:
String length is 44 Pass count is 1000
horsch1 : 0.77517914772
horsch2 : 0.778718948364
jreduce : 0.0403778553009
jcounter: 0.0699260234833
String length is 4931 Pass count is 100
horsch1 : 8.25176692009
horsch2 : 8.14318394661
jreduce : 0.260674953461
jcounter: 0.282369852066
(reduce方法的运行速度略快于Counter方法。)
计时代码如下。它使用模块。在这里的代码中,timeit.Timer
的第一个参数是要重复计时的代码,第二个参数是设置代码
import timeit
from collections import Counter
passes = 1000
m1 = lambda x: [int(ord(x) == i) for i in xrange(65,91)]
def m2(x):
return [int(ord(x) == i) for i in xrange(65,91)]
def es1(s):
add = lambda x,y: [x[i]+y[i] for i in xrange(len(x))]
freq = reduce(add,map(m1, s.upper()))
return freq
def es2(s):
add = lambda x,y: [x[i]+y[i] for i in xrange(len(x))]
freq = reduce(add,map(m2, s.upper()))
return freq
def addto(d,x):
d[x] = d.get(x,0) + 1
return d
def jwc(s):
return Counter(s)
def jwr(s):
return reduce (addto, s, {})
s = "the quick brown fox jumped over the lazy dog"
print 'String length is',len(s), ' Pass count is',passes
print "horsch1 :",timeit.Timer('f(s)', 'from __main__ import s, m1, es1 as f').timeit(passes)
print "horsch2 :",timeit.Timer('f(s)', 'from __main__ import s, m2, es2 as f').timeit(passes)
print "jreduce :",timeit.Timer('f(s)', 'from __main__ import s, addto, jwr as f').timeit(passes)
print "jcounter:",timeit.Timer('f(s)', 'from __main__ import s, Counter,jwc as f').timeit(passes)
ord()
通常给出ascii码。我的方法计算每个索引对应于字母表中该位置的字母的频率。由于是上套管,所以该方法不区分大小写
s = "the quick brown fox jumped over the lazy dog"
# Map function
m = lambda x: [ord(x) == i for i in xrange(0,26)]
add = lambda x,y: [x[i]+y[i] for i in xrange(len(x))]
freq = reduce(add,map(m, s.upper()))
您还可以使用
defaultdict
:
>>> from collections import defaultdict
>>> d = defaultdict(int)
>>> s = 'the quick brown fox jumped over the lazy dog'
>>> for i in s:
... d[i] += 1
...
>>> for letter,count in d.iteritems():
... print letter,count
...
8 # number of spaces
a 1
c 1
b 1
e 4
d 2
g 1
f 1
i 1
h 2
k 1
j 1
m 1
l 1
o 4
n 1
q 1
p 1
r 2
u 2
t 2
w 1
v 1
y 1
x 1
z 1
您还可以使用s.count方法:
{x: s.count(x) for x in set(s)}
请注意,我使用set
只计算字符串中每个字母的频率一次。这是在我的机器上测试的结果:
String length is 44 Pass count is 1000
horsch1 : 0.317646980286
horsch2 : 0.325616121292
jreduce : 0.0106990337372
jcounter : 0.0142340660095
def_dict : 0.00750803947449
just_dict: 0.00737881660461
s_count : 0.00887513160706
String length is 4400 Pass count is 100
horsch1 : 3.24123382568
horsch2 : 3.23079895973
jreduce : 0.0944828987122
jcounter : 0.102299928665
def_dict : 0.0341360569
just_dict: 0.0643239021301
s_count : 0.0224709510803
这是一个测试代码:
import timeit
from collections import Counter, defaultdict
passes = 100
m1 = lambda x: [int(ord(x) == i) for i in xrange(65,91)]
def m2(x):
return [int(ord(x) == i) for i in xrange(65,91)]
def es1(s):
add = lambda x,y: [x[i]+y[i] for i in xrange(len(x))]
freq = reduce(add,map(m1, s.upper()))
return freq
def es2(s):
add = lambda x,y: [x[i]+y[i] for i in xrange(len(x))]
freq = reduce(add,map(m2, s.upper()))
return freq
def addto(d,x):
d[x] = d.get(x,0) + 1
return d
def jwc(s):
return Counter(s)
def jwr(s):
return reduce (addto, s, {})
def def_dict(s):
d = defaultdict(int)
for i in s:
d[i]+=1
return d
def just_dict(s):
freq = {}
for i in s:
freq[i]=freq.get(i, 0) + 1
return freq
def s_count(s):
return {x: s.count(x) for x in set(s)}
s = "the quick brown fox jumped over the lazy dog"*100
print 'String length is',len(s), ' Pass count is',passes
print "horsch1 :",timeit.Timer('f(s)', 'from __main__ import s, m1, es1 as f').timeit(passes)
print "horsch2 :",timeit.Timer('f(s)', 'from __main__ import s, m2, es2 as f').timeit(passes)
print "jreduce :",timeit.Timer('f(s)', 'from __main__ import s, addto, jwr as f').timeit(passes)
print "jcounter :",timeit.Timer('f(s)', 'from __main__ import s, Counter,jwc as f').timeit(passes)
print "def_dict :",timeit.Timer('f(s)', 'from __main__ import s, defaultdict, def_dict as f').timeit(passes)
print "just_dict:",timeit.Timer('f(s)', 'from __main__ import s, just_dict as f').timeit(passes)
print "s_count :",timeit.Timer('f(s)', 'from __main__ import s, s_count as f').timeit(passes)
你的addto解决方案很好。我真的很喜欢它。我试着用一些肮脏的东西在lambda内部做这件事-我想跳出盒子是更好的做法:)很好的解决方案,+1。出于好奇,你的addto(d,x)解决方案的效率与我在下面写的解决方案相比如何?@emschorsch,见编辑。你可以对计时代码进行更改,以查看时间的去向。哇!谢谢你说明我的方法有多慢。我很难想出一个使用map和reduce的方法,所以我觉得我的代码很好,因为它看起来相当简洁。如果你用
[x==i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=i=2/3。(还要注意add=…行中缺少]我不知道可以在python中添加布尔值并获得整数和。为什么“字母表”中i的要比xrange(0,25)
中i的快?我不知道实现细节,但想象一下,在遍历字符串时,这可能会降低开销(例如,节省更少的上下文)。可能int(ord(x)==i)
更重要。在编译语言中,int(ord(x)==i)
和x==i
具有相同的低级代码。但是在python中,int和ord需要时间来执行。