如何在Python中可靠地处理web数据
我使用以下代码从网站获取数据:如何在Python中可靠地处理web数据,python,web,urllib2,urlopen,Python,Web,Urllib2,Urlopen,我使用以下代码从网站获取数据: time_out = 4 def tryconnect(turl, timer=time_out, retries=10): urlopener = None sitefound = 1 tried = 0 while (sitefound != 0) and tried < retries: try: urlopener = urllib2.urlopen(turl, None, t
time_out = 4
def tryconnect(turl, timer=time_out, retries=10):
urlopener = None
sitefound = 1
tried = 0
while (sitefound != 0) and tried < retries:
try:
urlopener = urllib2.urlopen(turl, None, timer)
sitefound = 0
except urllib2.URLError:
tried += 1
if urlopener: return urlopener
else: return None
有没有一种方法可以处理所有这些异常,而不必每次都有那么多样板代码
谢谢 在第一个函数中也可以避免一些样板代码:
time_out = 4
def tryconnect(turl, timer=time_out, retries=10):
for tried in xrange(retries):
try:
return urllib2.urlopen(turl, None, timer)
except urllib2.URLError:
pass
return None
在第二种情况下:
urlopener = tryconnect('www.example.com')
if urlopener:
try:
for line in urlopener:
do stuff
except (httplib.IncompleteRead, socket.timeout), e:
print e
return None
else:
return None
您也可以在第一个函数中避免一些样板代码:
time_out = 4
def tryconnect(turl, timer=time_out, retries=10):
for tried in xrange(retries):
try:
return urllib2.urlopen(turl, None, timer)
except urllib2.URLError:
pass
return None
在第二种情况下:
urlopener = tryconnect('www.example.com')
if urlopener:
try:
for line in urlopener:
do stuff
except (httplib.IncompleteRead, socket.timeout), e:
print e
return None
else:
return None
感谢来自C的答案(你可能已经猜到,使用0作为false:)我正在慢慢学习如何编写pythonic代码。感谢来自C的答案(你可能已经猜到,使用0作为false:)我正在慢慢学习如何编写pythonic代码。