如何使Python请求在获取head时持久化

我有一个问题，Python请求在几秒钟后抛出异常。网站运行非常缓慢，但只是间歇性的

Chrome和Safari都无法加载页面。（例如，Chrome显示“无法访问此站点”、连接错误或重置）。然而，Firefox始终能够访问该页面，尽管加载大约需要20秒。这种行为可以在位于不同国家/地区的多台不同机器上重复。看起来Firefox“更努力”，而不是超时

在这种情况下，我希望Python的请求表现得更像Firefox。我已经将

timeout

参数设置为一个大数字（60秒），但在此之前很久就会抛出异常。似乎存在某种握手超时，而

timeout

参数可能控制响应的等待时间，即握手后

import requests
target='https://nomads.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.20191113/00/pgrb2a/'
request = requests.head(target, timeout=60)
print(request.status_code) 

^^将

20191113

替换为昨天的日期，因为这些链接在7天后过期

异常大约在5秒后到达，并且是无法访问页面时的“标准”请求异常：

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    599                                                   body=body, headers=headers,
--> 600                                                   chunked=chunked)
    601 

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    383                     # otherwise it looks like a programming error was the cause.
--> 384                     six.raise_from(e, None)
    385         except (SocketTimeout, BaseSSLError, SocketError) as e:

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/packages/six.py in raise_from(value, from_value)

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    379                 try:
--> 380                     httplib_response = conn.getresponse()
    381                 except Exception as e:

~/miniconda/envs/basics/lib/python3.6/http/client.py in getresponse(self)
   1330             try:
-> 1331                 response.begin()
   1332             except ConnectionError:

~/miniconda/envs/basics/lib/python3.6/http/client.py in begin(self)
    296         while True:
--> 297             version, status, reason = self._read_status()
    298             if status != CONTINUE:

~/miniconda/envs/basics/lib/python3.6/http/client.py in _read_status(self)
    257     def _read_status(self):
--> 258         line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
    259         if len(line) > _MAXLINE:

~/miniconda/envs/basics/lib/python3.6/socket.py in readinto(self, b)
    585             try:
--> 586                 return self._sock.recv_into(b)
    587             except timeout:

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py in recv_into(self, *args, **kwargs)
    299             else:
--> 300                 return self.recv_into(*args, **kwargs)
    301 

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py in recv_into(self, *args, **kwargs)
    289             else:
--> 290                 raise SocketError(str(e))
    291         except OpenSSL.SSL.ZeroReturnError as e:

OSError: (54, 'ECONNRESET')

During handling of the above exception, another exception occurred:

ProtocolError                             Traceback (most recent call last)
~/miniconda/envs/basics/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    444                     retries=self.max_retries,
--> 445                     timeout=timeout
    446                 )

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    637             retries = retries.increment(method, url, error=e, _pool=self,
--> 638                                         _stacktrace=sys.exc_info()[2])
    639             retries.sleep()

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
    366             if read is False or not self._is_method_retryable(method):
--> 367                 raise six.reraise(type(error), error, _stacktrace)
    368             elif read is not None:

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/packages/six.py in reraise(tp, value, tb)
    684         if value.__traceback__ is not tb:
--> 685             raise value.with_traceback(tb)
    686         raise value

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    599                                                   body=body, headers=headers,
--> 600                                                   chunked=chunked)
    601 

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    383                     # otherwise it looks like a programming error was the cause.
--> 384                     six.raise_from(e, None)
    385         except (SocketTimeout, BaseSSLError, SocketError) as e:

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/packages/six.py in raise_from(value, from_value)

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    379                 try:
--> 380                     httplib_response = conn.getresponse()
    381                 except Exception as e:

~/miniconda/envs/basics/lib/python3.6/http/client.py in getresponse(self)
   1330             try:
-> 1331                 response.begin()
   1332             except ConnectionError:

~/miniconda/envs/basics/lib/python3.6/http/client.py in begin(self)
    296         while True:
--> 297             version, status, reason = self._read_status()
    298             if status != CONTINUE:

~/miniconda/envs/basics/lib/python3.6/http/client.py in _read_status(self)
    257     def _read_status(self):
--> 258         line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
    259         if len(line) > _MAXLINE:

~/miniconda/envs/basics/lib/python3.6/socket.py in readinto(self, b)
    585             try:
--> 586                 return self._sock.recv_into(b)
    587             except timeout:

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py in recv_into(self, *args, **kwargs)
    299             else:
--> 300                 return self.recv_into(*args, **kwargs)
    301 

~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py in recv_into(self, *args, **kwargs)
    289             else:
--> 290                 raise SocketError(str(e))
    291         except OpenSSL.SSL.ZeroReturnError as e:

ProtocolError: ('Connection aborted.', OSError("(54, 'ECONNRESET')",))

During handling of the above exception, another exception occurred:

ConnectionError                           Traceback (most recent call last)
<ipython-input-2-e4852eeb80e3> in <module>()
      2 import requests
      3 target='https://nomads.ncep.noaa.gov/pub/data/nccf/com/gens/prod/gefs.20191113/00/pgrb2a/'
----> 4 request = requests.head(target, timeout=60)
      5 print(request.status_code)

~/miniconda/envs/basics/lib/python3.6/site-packages/requests/api.py in head(url, **kwargs)
     96 
     97     kwargs.setdefault('allow_redirects', False)
---> 98     return request('head', url, **kwargs)
     99 
    100 

~/miniconda/envs/basics/lib/python3.6/site-packages/requests/api.py in request(method, url, **kwargs)
     56     # cases, and look like a memory leak in others.
     57     with sessions.Session() as session:
---> 58         return session.request(method=method, url=url, **kwargs)
     59 
     60 

~/miniconda/envs/basics/lib/python3.6/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    510         }
    511         send_kwargs.update(settings)
--> 512         resp = self.send(prep, **send_kwargs)
    513 
    514         return resp

~/miniconda/envs/basics/lib/python3.6/site-packages/requests/sessions.py in send(self, request, **kwargs)
    620 
    621         # Send the request
--> 622         r = adapter.send(request, **kwargs)
    623 
    624         # Total elapsed time of the request (approximately)

~/miniconda/envs/basics/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    493 
    494         except (ProtocolError, socket.error) as err:
--> 495             raise ConnectionError(err, request=request)
    496 
    497         except MaxRetryError as e:

ConnectionError: ('Connection aborted.', OSError("(54, 'ECONNRESET')",))

---------------------------------------------------------------------------
OSError回溯（最近一次调用上次）
urlopen中的~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py（self、方法、url、正文、标题、重试、重定向、断言相同的主机、超时、池超时、释放连接、分块、正文位置、**响应）
599主体=主体，标题=标题，
-->600分块=分块）
601
~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py在发出请求（self、conn、method、url、timeout、chunked、**httplib\u request\u kw）
383#否则看起来是编程错误造成的。
-->384六.从（e，无）处升起
385除（SocketTimeout、BaseSLerror、SocketError）外，如e：
在raise\u from（value，from\u value）中的~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/packages/six.py
~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py在发出请求（self、conn、method、url、timeout、chunked、**httplib\u request\u kw）
379试试：
-->380 httplib_response=conn.getresponse（）
381例外情况除外，如e：
getresponse（self）中的~/miniconda/envs/basics/lib/python3.6/http/client.py
1330尝试：
->1331响应。开始（）
1332除连接错误外：
begin（self）中的~/miniconda/envs/basics/lib/python3.6/http/client.py
296虽然正确：
-->297版本，状态，原因=self.\u读取\u状态（）
298如果状态！=继续：
~/miniconda/envs/basics/lib/python3.6/http/client.py处于读取状态（self）
257 def_读取_状态（自身）：
-->258行=str（self.fp.readline（_MAXLINE+1），“iso-8859-1”）
259如果长度（线）>最大线：
readinto（self，b）中的~/miniconda/envs/basics/lib/python3.6/socket.py
585试试：
-->586返回自存储到（b）中
587除超时外：
将~/miniconda/envs/basics/lib/python3.6/site-packages/urlib3/contrib/pyopenssl.py放入（self，*args，**kwargs）
299其他：
-->300将self.recv_返回（*args，**kwargs）
301
将~/miniconda/envs/basics/lib/python3.6/site-packages/urlib3/contrib/pyopenssl.py放入（self，*args，**kwargs）
289其他：
-->290升高插座错误（str（e））
291除OpenSSL.SSL.ZeroReturn错误外，错误为e:
OSError:（54，‘经济复苏’）
在处理上述异常期间，发生了另一个异常：
协议错误回溯（最后一次最近调用）
发送中的~/miniconda/envs/basics/lib/python3.6/site-packages/requests/adapters.py（self、request、stream、timeout、verify、cert、proxies）
444次重试=自最大重试次数，
-->445超时=超时
446                 )
urlopen中的~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py（self、方法、url、正文、标题、重试、重定向、断言相同的主机、超时、池超时、释放连接、分块、正文位置、**响应）
637 retries=retries.increment（方法，url，错误=e，_pool=self，
-->638 _stacktrace=sys.exc_info（）[2]）
639次重试。sleep（）
增量中的~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/util/retry.py（self、方法、url、响应、错误、池、堆栈跟踪）
366如果读取为False或非self.\u是\u方法\u可重试（方法）：
-->367升起六个。重新升起（类型（错误），错误，_stacktrace）
368 elif read不是无：
reraise中的~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/packages/six.py（tp、value、tb）
684如果值.\uuuu回溯\uuuuuu不是tb：
-->685带回溯（tb）的提升值
686提高价值
urlopen中的~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py（self、方法、url、正文、标题、重试、重定向、断言相同的主机、超时、池超时、释放连接、分块、正文位置、**响应）
599主体=主体，标题=标题，
-->600分块=分块）
601
~/miniconda/envs/basics/lib/python3.6/site-packages/urllib3/connectionpool.py在发出请求（self、conn、method、url、timeout、chunked、**httplib\u request\u kw）
383#否则看起来是编程错误造成的。
-->384六.从（e，无）处升起
385除（SocketTimeout、BaseSLerror、SocketError）外，如e：
~/
@backoff.on_exception(backoff.expo,
                      requests.exceptions.RequestException,
                      max_time=60)
def get_url(url):
    return requests.get(url)