如何在Python中可靠地处理web数据

如何在Python中可靠地处理web数据,python,web,urllib2,urlopen,Python,Web,Urllib2,Urlopen,我使用以下代码从网站获取数据: time_out = 4 def tryconnect(turl, timer=time_out, retries=10): urlopener = None sitefound = 1 tried = 0 while (sitefound != 0) and tried < retries: try: urlopener = urllib2.urlopen(turl, None, t

我使用以下代码从网站获取数据:

time_out = 4

def tryconnect(turl, timer=time_out, retries=10):
    urlopener = None
    sitefound = 1
    tried = 0
    while (sitefound != 0) and tried < retries:
        try:
            urlopener = urllib2.urlopen(turl, None, timer)
            sitefound = 0
        except urllib2.URLError:
            tried += 1
    if urlopener: return urlopener
    else: return None
有没有一种方法可以处理所有这些异常,而不必每次都有那么多样板代码


谢谢

在第一个函数中也可以避免一些样板代码:

time_out = 4

def tryconnect(turl, timer=time_out, retries=10):
    for tried in xrange(retries):
        try:
            return urllib2.urlopen(turl, None, timer)
        except urllib2.URLError:
            pass
    return None
在第二种情况下:

urlopener = tryconnect('www.example.com')
if urlopener:
    try:
        for line in urlopener:
            do stuff
    except (httplib.IncompleteRead, socket.timeout), e:
        print e
        return None
else:
    return None

您也可以在第一个函数中避免一些样板代码:

time_out = 4

def tryconnect(turl, timer=time_out, retries=10):
    for tried in xrange(retries):
        try:
            return urllib2.urlopen(turl, None, timer)
        except urllib2.URLError:
            pass
    return None
在第二种情况下:

urlopener = tryconnect('www.example.com')
if urlopener:
    try:
        for line in urlopener:
            do stuff
    except (httplib.IncompleteRead, socket.timeout), e:
        print e
        return None
else:
    return None

感谢来自C的答案(你可能已经猜到,使用0作为false:)我正在慢慢学习如何编写pythonic代码。感谢来自C的答案(你可能已经猜到,使用0作为false:)我正在慢慢学习如何编写pythonic代码。