Python 遍历数组并在文件中搜索数组中的每个项
我甚至不知道我问这个问题的方式是否正确,但我想搜索一个日志文件并查找数组中的每个单词。此时,我已要求用户将有问题的文件拖到终端中,然后用输入构建一个数组。程序应该打印出每个单词所在的行 一旦我开始工作,我会格式化,有一个计数器,或者对我在文件中找到的东西做一个小结,等等 这是我到目前为止得到的,只是当我运行它时,它实际上找不到任何单词。我一直在查看重复使用示例,但我认为对于我的想法来说可能过于复杂:Python 遍历数组并在文件中搜索数组中的每个项,python,search,file-io,Python,Search,File Io,我甚至不知道我问这个问题的方式是否正确,但我想搜索一个日志文件并查找数组中的每个单词。此时,我已要求用户将有问题的文件拖到终端中,然后用输入构建一个数组。程序应该打印出每个单词所在的行 一旦我开始工作,我会格式化,有一个计数器,或者对我在文件中找到的东西做一个小结,等等 这是我到目前为止得到的,只是当我运行它时,它实际上找不到任何单词。我一直在查看重复使用示例,但我认为对于我的想法来说可能过于复杂: def wordsToFind(): needsWords = True sea
def wordsToFind():
needsWords = True
searchArray = []
print "Add words to search ('done') to save/continue."
while needsWords == True:
word = raw_input("Enter a search word: ")
if word.lower() == "done":
needsWords = False
break
else:
searchArray.append(word)
print word + " added"
return searchArray
def getFile():
file_to_read = raw_input("Drag file here:").strip()
return file_to_read
def main():
filePath = getFile()
searchArray = wordsToFind()
print "Words searched for: ", searchArray
searchCount = []
with open(filePath, "r") as inFile:
for line in inFile:
for item in searchArray:
if item in line:
print item
main()
显然,这里强烈欢迎任何优化建议或更好的python编码建议,我只知道我所知道的,并感谢所有的帮助 您可以这样做:
a = ['foo', 'bar', 'cox', 'less', 'more']
b = ['foo', 'cox', 'complex', 'list']
c = list(set(a).intersection(set(b)))
这样,c将是:
['cox', 'foo']
实现这一点的另一种方法是使用python理解:
c = [x for x in a if x in b]
我不知道哪种方法最快,但我认为是使用集合…这正是map reduce想要解决的问题。如果您不熟悉,map reduce是一个简单的两步过程。假设您有一个列表,其中存储了您想在文本中查找的单词。映射器函数可以针对文本的每一行遍历该单词列表,如果它出现在该行中,它将返回一个值,例如,['word',lineNum],该值存储在结果列表中。映射器本质上是for循环上的包装器。然后,您可以通过编写一个reducer函数来获取结果列表并“缩减”它,在本例中,该函数可以将看起来像[['word1',1]…['word1',n]…]的结果列表放入一个看起来像{'word1':[1,2,5],'word3':[7],…]的对象中 这种方法是有利的,因为您在对每个项执行公共操作时抽象了迭代列表的过程,并且如果您的分析需要更改(就像它们经常做的那样),您只需要更改映射器/缩减函数,而不必涉及代码的其余部分。此外,这个方法是高度并行的,如果它成为一个问题的话(问问谷歌吧!) Python3.x具有内置的map/reduce方法,如map()和reduce();在python文档中查找它们。因此,您可以看到它们是如何工作的,我根据您的问题实现了一个版本的map/reduce,而不使用内置库。因为您没有指定数据的存储方式,所以我对它做了一些假设,即感兴趣的单词列表将作为逗号分隔的文件给出。为了读取文本文件,我使用readlines()获取一个行数组,并使用正则表达式模式将行拆分为单词(即,拆分任何不是字母数字字符的内容)。当然,这可能不适合您的需要,因此您可以将其更改为对正在查看的文件有意义的任何内容 我试图避开深奥的python特性(没有lambdas!),因此希望实现是清晰的。最后一个注意事项是,我使用了一个循环来迭代文本文件的行,使用了一个map函数来迭代感兴趣的单词列表。您可以使用嵌套的映射函数,但我希望跟踪循环索引(因为您关心行号)。如果确实要嵌套map函数,可以在读取文件时将数组行存储为line和line number的元组,也可以修改map函数以返回索引,这取决于您的选择 我希望这有帮助
#!usr/bin/env/ python
#Regexp library
import re
#Map
#This function returns a new array containing
#the elements after that have been modified by whatever function we passed in.
def mapper(function, sequence):
#List to store the results of the map operation
result = []
#Iterate over each item in sequence, append the values to the results list
#after they have been modified by the "function" supplied as an argument in the
#mapper function call.
for item in sequence:
result.append(function(item))
return result
#Reduce
#The purpose of the reduce function is to go through an array, and combine the items
#according to a specified function - this specified function should combine an element
#with a base value
def reducer(function, sequence, base_value):
#Need to get an base value to serve as the starting point for the construction of
#the result
#I will assume one is given, but in most cases you should include extra validation
#here to either ensure one is given, or some sensible default is chosen
#Initialize our accumulative value object with the base value
accum_value = base_value
#Iterate through the sequence items, applying the "function" provided, and
#storing the results in the accum_value object
for item in sequence:
accum_value = function(item, accum_value)
return accum_value
#With these functions it should be sufficient to address your problem, what remains
#is simply to get the data from the text files, and keep track of the lines in
#which words appear
if __name__ == 'main':
word_list_file = 'FILEPATH GOES HERE'
#Read in a file containing the words that will be searched in the text file
#(assumes words are given as a comma separated list)
infile = open(word_list_file, 'rt') #Open file
content = infile.read() #read the whole file as a single string
word_list = content.split(',') #split the string into an array of words
infile.close()
target_text_file = 'FILEPATH GOES HERE'
#Read in the text to analyze
infile = open(target_text_file, 'rt') #Open file
target_text_lines = infile.readlines() #Read the whole file as an array of lines
infile.close()
#With the data loaded, the overall strategy will be to loop over the text lines, and
#we will use the map function to loop over the the word_list and see if they are in
#the current text file line
#First, define the my_mapper function that will process your data, and will be passed to
#the map function
def my_mapper(item):
#Split the current sentence into words
#Will split on any non alpha-numeric character. This strategy can be revised
#to find matches to a regular expression pattern based on the words in the
#words list. Either way, make sure you choose a sensible strategy to do this.
current_line_words = re.split(r'\W+', target_text_lines[k])
#lowercase the words
current_line_words = [word.lower() for word in current_line_words]
#Check if the current item (word) is in the current_line_words list, and if so,
#return the word and the line number
if item in current_line_words:
return [item, k+1] #Return k+1 because k begins at 0, but I assume line
#counting begins with 1?
else:
return [] #Technically, this does not need to be added, it can simply
#return None by default, but that requires manually handling iterator
#objects so the loop doesn't crash when seeing the None values,
#and I am being lazy :D
#With the mapper function established, we can proceed to loop over the text lines of the
#array, and use our map function to process the lines against the list of words.
#This array will store the results of the map operation
map_output = []
#Loop over text file lines, use mapper to find which words are in which lines, store
#in map_output list. This is the exciting stuff!
for k in range(len(target_text_lines)):
map_output.extend(mapper(my_mapper, word_list))
#At this point, we should have a list of lists containing the words and the lines they
#appeared in, and it should look like, [['word1', 1] ... ['word25': 5] ... [] ...]
#As you can see, the post-map array will have an entry for each word that appeared in
#each line, and if a particular word did not appear in a particular line, there will be a
#empty list instead.
#Now all that remains is to summarize our data, and that is what the reduce function is
#for. We will iterate over the map_output list, and collect the words and which lines
#they appear at in an object that will have the format { 'word': [n1, n2, ...] },where
#n1, n2, ... are the lines the word appears in. As in the case for the mapper
#function, the output of the reduce function can be modified in the my_reducer function
#you supply to it. If you'd rather it return something else (like say, word count), this
#is the function to modify.
def my_reducer(item, accum_value):
#First, verify item is not empty
if item != []:
#If the element already exists in the output object, append the current line
#value to it, if not, add it to the object and create a set holding the current
#line value
#Check this word/line combination isn't already stored in the output dict
if (item[0] in accum_value) and (item[1] not in accum_value[item[0]]):
accum_value[item[0]].append(item[1])
else:
accum_value[item[0]] = [item[1]]
return accum_value
#Now we can call the reduce function, save it's output, print it to screen, and we're
#done!
#(Note that for base value we are just passing in an empty object, {})
reduce_results = reducer(my_reducer, map_output, {})
#Print results to screen
for result in reduce_results:
print('word: {}, lines: {}'.format(result, reduce_results[result]))
看一看内置的
any
。你能举个例子说明它什么都找不到吗?你的代码对我来说似乎工作得很好(虽然你的搜索逻辑可以改进)。嘿,这无疑为我指明了正确的方向,不过如果你不介意的话,我有一个问题。我想使用列表理解方法(学习目的),我制作了:b=open(filePath,“r”).readlines()我已经打印了内容,它看起来像是把每一行作为列表中的一个项目来保存,这非常棒,但是。。。found=[x代表searchArray中的x,如果b中的x]未填充任何内容。另外,如果我现在在两个数组中找到了交集,那么如何打印整行?多谢!关于我没有理解的问题,你能提供代码吗?对于交叉点,您想打印新列表,还是列表中的每个元素?是的,这很好。我刚刚为regex完成了Googledev课程,这真是太棒了。谢谢你帮我把这个放在一起,它工作得很好。