Python ConnectionResetError with requests模块在异常内部引发异常,导致程序崩溃

Python ConnectionResetError with requests模块在异常内部引发异常,导致程序崩溃,python,exception,web-scraping,python-requests,Python,Exception,Web Scraping,Python Requests,使我的程序崩溃的异常如下: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/response.py", line 397, in _error_catcher yield File "/Library/Frameworks/Python.framework/Versions/3

使我的程序崩溃的异常如下:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/response.py", line 397, in _error_catcher
    yield
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/response.py", line 479, in read
    data = self._fp.read(amt)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 447, in read
    n = self.readinto(b)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 491, in readinto
    n = self.fp.readinto(b)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 54] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/models.py", line 750, in generate
    for chunk in self.raw.stream(chunk_size, decode_content=True):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/response.py", line 531, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/response.py", line 496, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 130, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/response.py", line 415, in _error_catcher
    raise ProtocolError('Connection broken: %r' % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(54, 'Connection reset by peer')", ConnectionResetError(54, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/me/Documents/CBB/scrape.py", line 301, in <module>
    main([2018, 11, 6, 2018, 12, 1])
  File "/Users/me/Documents/CBB/scrape.py", line 207, in main
    get_range(int(argv[0]), int(argv[1]), int(argv[2]), int(argv[3]), int(argv[4]), int(argv[5]))
  File "/Users/me/Documents/CBB/scrape.py", line 250, in get_range
    rg.get_bsc()
  File "/Users/me/Documents/CBB/scrape.py", line 103, in get_bsc
    self.get_pbp()
  File "/Users/me/Documents/CBB/scrape.py", line 120, in get_pbp
    soup = self.scraper.open_page(url=url)
  File "/Users/me/Documents/CBB/scrape_util.py", line 209, in open_page
    response = self.session.get(url, proxies={'https': ip, 'http': ip}, headers=headers)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 686, in send
    r.content
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/models.py", line 828, in content
    self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/models.py", line 753, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(54, 'Connection reset by peer')", ConnectionResetError(54, 'Connection reset by peer'))
回溯(最近一次呼叫最后一次):
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site packages/urllib3/response.py”,第397行,在“错误捕捉器”中
产量
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site packages/urllib3/response.py”,第479行,已读
数据=自计量读数(金额)
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py”,第447行,已读
n=自读入(b)
readinto中的文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py”,第491行
n=自fp读入(b)
readinto中的文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py”,第589行
返回自我。将袜子重新放入(b)
ConnectionResetError:[Errno 54]对等方重置连接
在处理上述异常期间,发生了另一个异常:
回溯(最近一次呼叫最后一次):
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site packages/requests/models.py”,第750行,在generate中
对于self.raw.stream中的块(块大小,解码内容=True):
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site packages/urllib3/response.py”,第531行,在流中
数据=自读(金额=金额,解码内容=解码内容)
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site packages/urllib3/response.py”,第496行,已读
提升未完成读取(self.\u fp\u字节\u读取,self.length\u剩余)
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py”,第130行,在退出时__
self.gen.throw(类型、值、回溯)
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site packages/urllib3/response.py”,第415行,在错误捕获器中
引发协议错误('连接断开:%r'%e,e)
urllib3.exceptions.ProtocolError:(“连接断开:ConnectionResetError(54,‘由对等方重置连接’)”,ConnectionResetError(54,‘由对等方重置连接’))
在处理上述异常期间,发生了另一个异常:
回溯(最近一次呼叫最后一次):
文件“/Users/me/Documents/CBB/scrap.py”,第301行,在
主要([2018,11,6,2018,12,1])
文件“/Users/me/Documents/CBB/scrap.py”,第207行,在main中
获取范围(int(argv[0])、int(argv[1])、int(argv[2])、int(argv[3])、int(argv[4])、int(argv[5]))
文件“/Users/me/Documents/CBB/scrap.py”,第250行,在get\u范围内
rg.get_bsc()
文件“/Users/me/Documents/CBB/scrap.py”,get_bsc中的第103行
self.get_pbp()
文件“/Users/me/Documents/CBB/scrap.py”,第120行,在get_pbp中
soup=self.scraper.open_页面(url=url)
文件“/Users/me/Documents/CBB/scrape_util.py”,第209行,在打开的页面中
response=self.session.get(url,代理={'https':ip,'http':ip},headers=headers)
get中的文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site packages/requests/sessions.py”,第546行
返回self.request('GET',url,**kwargs)
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site packages/requests/sessions.py”,请求中第533行
resp=自我发送(准备,**发送)
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site packages/requests/sessions.py”,第686行,在send中
r、 内容
content中的文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site packages/requests/models.py”,第828行
self.\u content=b“”。加入(self.iter\u content(content\u CHUNK\u SIZE))或b“”
文件“/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site packages/requests/models.py”,第753行,在generate中
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError:(“连接断开:ConnectionResetError(54,‘对等方重置连接’)”,ConnectionResetError(54,‘对等方重置连接’))
我在一个网站上运行一个webscraper,在这个网站上,互联网偶尔会一次中断一两秒钟,在这种情况下,我希望会收到某种例外。我处理这些异常的方法是等待几秒钟,然后再次尝试访问页面。对于此问题导致的大多数类型的异常,我可以轻松地
排除异常并重试。但是,每当这导致出现
ConnectionResetError
时,程序就会因为异常处理程序中引发的异常而崩溃。我不知道如何解决这个问题,因为引发任何异常的代码都不是我写的。我没有提出任何例外,我想扼杀所有这些,以便我可以重试。有没有办法在异常崩溃中忽略这些异常


我已经排除了
ConnectionResetError
s,并尽快离开它们,但这不起作用。我正在使用“请求”模块打开这些页面。

这里有3种不同的例外情况,它们按以下顺序出现:

  • ConnectionResetError
  • urllib3.exceptions.ProtocolError
  • requests.exceptions.ChunkedEncodingError
  • 前两个异常在您正在使用的库中是例外的,您不应该关心它们。最后一个(
    requests.exceptions.ChunkedEncodingError
    )不被排除,因为您也没有排除它,所以引发了它。因此,你应该:

    except requests.exceptions.ChunkedEncodingError:
        # retry like you do for other exceptions
    
    要说明发生了什么,请检查以下代码:

    def external_function():
        try:
            1 / 0    #  this produces ZeroDivisionError
        except ZeroDivisionError:
            a = b    #  this produces NameError
    
    try:
        external_function()
    except ZeroDivisionError:
        print('This line is never printed')
    except NameError:
        print('NamerError is caught')
    
    external\u function()
    演示了
    请求。get()
    。首先,
    ZeroDivisionError
    错误发生。但是,不会引发它,因为它是由一些外部代码处理的。在处理此异常期间,外部代码会产生另一个异常,这次是
    namererror
    。此异常不由外部代码处理,因此将返回给您。那就由你来处理了。如果你不处理