处理python请求的异常
我的函数是通过线程执行的:处理python请求的异常,python,python-requests,Python,Python Requests,我的函数是通过线程执行的: def getdata(self, page, ...): tries = 10 for n in range(tries): try: ... datarALL = [] url = 'http://website/...'.format(...) responsedata = requests.get(url, data=data, he
def getdata(self, page, ...):
tries = 10
for n in range(tries):
try:
...
datarALL = []
url = 'http://website/...'.format(...)
responsedata = requests.get(url, data=data, headers=self.hed, verify=False)
responsedata.raise_for_status()
if responsedata.status_code == 200: # 200 for successful call
...
if ...
break
except (ChunkedEncodingError, requests.exceptions.HTTPError) as e:
print ("page #{0} run #{1} failed. Returned status code {2}. Reason: {3}. Msg: {4}. Retry.".format(page, n, responsedata.status_code, responsedata.reason, sys.exc_info()[0]))
if n == tries - 1:
print ("page {0} could not be imported. Max retried reached.".format(page))
os._exit(1) #One thread max retried - close all threads and
return datarALL
详情如下:
with ThreadPoolExecutor(max_workers=num_of_workers) as executor:
futh = [(executor.submit(self.getdata, page,...)) for page in pages]
for data in as_completed(futh):
datarALL.extend(data.result())
print ("Finished generateing data.")
return datarALL
有时我会遇到意外的异常,例如:ConnectionResetError:[Errno 104]对等方重置连接
,这会关闭我的程序。我想更改代码,这样无论发生何种异常,线程都会一直重试,直到满足的
。我不希望我的线程由于随机异常而关闭
我阅读了,但我不知道如何在不手动列出所有异常的情况下捕获所有异常。有没有一种通用的方法可以做到这一点
基本上我想要的是:
except (ALL EXCEPTIONS from Requests) as e:
print ("page #{0} run #{1} failed. Returned status code {2}. Reason: {3}. Msg: {4}. Retry.".format(page, n, responsedata.status_code, responsedata.reason, sys.exc_info()[0]))
if n == tries - 1:
print ("page {0} could not be imported. Max retried reached.".format(page))
os._exit(1) #One thread max retried - close all threads and
return datarALL
我该怎么做
编辑:
使用
不流行。它给了我这个:
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.5/site-packages/urllib3/response.py", line 331, in _error_catcher
yield
File "/home/ubuntu/.local/lib/python3.5/site-packages/urllib3/response.py", line 640, in read_chunked
chunk = self._handle_chunk(amt)
File "/home/ubuntu/.local/lib/python3.5/site-packages/urllib3/response.py", line 595, in _handle_chunk
returned_chunk = self._fp._safe_read(self.chunk_left)
File "/usr/lib/python3.5/http/client.py", line 607, in _safe_read
chunk = self.fp.read(min(amt, MAXAMOUNT))
File "/usr/lib/python3.5/socket.py", line 575, in readinto
return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer
循环不会重试。运行已终止
EDIT2:
except requests.exceptions.RequestException as e:
print ("page #{0} run #{1} failed. Returned status code {2}. Reason: {3}. Msg: {4}. Retry.".format(page, n, responsedata.status_code, responsedata.reason, sys.exc_info()[0]))
if n == tries - 1:
print ("page {0} could not be imported. Max retried reached.".format(page))
os._exit(1) #One thread max retried - close all threads and
return datarALL
也无法捕获上面列出的对等方的连接重置错误:[Errno 104]连接重置。捕获所有异常通常被认为是一种不好的做法,因为它可能隐藏一些问题 也就是说,Python异常从继承中受益,捕获一个基本异常将捕获从该基本异常继承的每个异常 有关详细信息,请参阅 您可以看到根异常是
BaseException
,但不应该捕获此异常,因为它将捕获Ctrl+C
中断和生成器退出。
如果您想捕获每种异常类型,您可以捕获异常
您还可以只捕获请求中的异常。在这种情况下,根据,它可以通过捕获请求
模块的基本异常来完成:请求异常
如果要捕获请求
异常和连接重置错误
(这是Python标准异常),必须在except
子句中指定这两个:
except (requests.exceptions.RequestException,
ConnectionResetError) as err:
# some code
或者,如果您希望不那么具体,并捕获所有可能的连接错误,您可以使用ConnectionError
而不是ConnectionResetError
。(见附件)
最后,您可能希望对每种异常类型做出不同的反应。在这种情况下,您可以执行以下操作:
try:
# something
except ConnectionError as err:
# manage connection errors
except requests.exceptions.RequestException as err:
# manage requests errors
检查您的设置.py
-是否在已安装的应用程序中添加了频道。除ConnectionResetError:………
?@BlackThunder我想捕获所有异常不仅仅是ConnectionErroruse除异常:
?这可能会帮助您@BlackThunder在我的问题中发布此链接。。请注意我写的关于它的内容,以明确我希望捕获请求包可以生成的所有异常。因为我不在乎原因是什么,所以我只想再试一次。只有在10次之后仍然存在异常时,我才想引发异常并停止代码。@Programmer120然后简单地捕获RequestException,如我在上一段中所做的那样:将RequestException作为e:
它给出:名称错误:未定义名称“RequestException”
将其更改为Exceptionrequests.exceptions.RequestException作为e:
现在它仍然无法捕获对等方重置的ConnectionResetError:[Errno 104]连接
为什么在请求包内不处理ConnectionError?
try:
# something
except ConnectionError as err:
# manage connection errors
except requests.exceptions.RequestException as err:
# manage requests errors