Python 需要帮助将我的类组合在一起吗

Python 需要帮助将我的类组合在一起吗,python,urllib2,Python,Urllib2,我正在尝试创建一个方法,该方法将更改urllib2.build_opener的用户代理 到目前为止,我得到的是: Crawler.py urlopener: 这就是我测试Crawler.py的方法: 在我的输出中,当前浏览器和新浏览器是相同的 Current Browser : Mozilla/5.0 (BlackBerry; U; BlackBerry 9850; en-US) AppleWebKit/534.11+ (KHTML, like Gecko) Version/7.0.0.115

我正在尝试创建一个方法,该方法将更改urllib2.build_opener的用户代理

到目前为止,我得到的是:

Crawler.py urlopener: 这就是我测试Crawler.py的方法: 在我的输出中,当前浏览器和新浏览器是相同的

Current Browser :
Mozilla/5.0 (BlackBerry; U; BlackBerry 9850; en-US) AppleWebKit/534.11+ (KHTML, like Gecko) Version/7.0.0.115 Mobile Safari/534.11+
New Browser:
Mozilla/5.0 (BlackBerry; U; BlackBerry 9850; en-US) AppleWebKit/534.11+ (KHTML, like Gecko) Version/7.0.0.115 Mobile Safari/534.11+
文件mob_brows.txt包含以下信息:

Mozilla/5.0 (Linux; U; Android 2.3.3; zh-tw; HTC_Pyramid Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
Mozilla/5.0 (Linux; U; Android 2.3.3; zh-tw; HTC_Pyramid Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari
Mozilla/5.0 (Linux; U; Android 2.3.3; zh-tw; HTC Pyramid Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
Mozilla/5.0 (Linux; U; Android 2.3.3; ko-kr; LG-LU3000 Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
Mozilla/5.0 (Linux; U; Android 2.3.3; en-us; HTC_DesireS_S510e Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile

你的代码是错误的,你将不得不重新制作它

我就是这样做我的网络爬虫的:

修改opener以接受用户代理作为参数

def opener(user_agent):
    #Process Hadlers
    opener=urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
    opener.addheaders=[
                    ('User-Agent', user_agent),
                # snip...
                ]
    return opener
然后生成具有不同用户代理字符串的开场白列表

# this could be nicer, but demonstrates the point
openers = [opener(agent) for agent in open('your_f_path')]
然后使用随机模块中的选项选择一个在爬虫类中分配self.opener=urlopener.opener的opener


为了更改用户代理,我需要在代码中更改什么
Mozilla/5.0 (Linux; U; Android 2.3.3; zh-tw; HTC_Pyramid Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
Mozilla/5.0 (Linux; U; Android 2.3.3; zh-tw; HTC_Pyramid Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari
Mozilla/5.0 (Linux; U; Android 2.3.3; zh-tw; HTC Pyramid Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
Mozilla/5.0 (Linux; U; Android 2.3.3; ko-kr; LG-LU3000 Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
Mozilla/5.0 (Linux; U; Android 2.3.3; en-us; HTC_DesireS_S510e Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile
def opener(user_agent):
    #Process Hadlers
    opener=urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
    opener.addheaders=[
                    ('User-Agent', user_agent),
                # snip...
                ]
    return opener
# this could be nicer, but demonstrates the point
openers = [opener(agent) for agent in open('your_f_path')]
from random import choice
use_to_open = choice(openers)