Python请求获取请求不一致性问题

您好，我对python和代理的使用还相当陌生，所以我非常感谢您的帮助！ 我正在做一些网页抓取和代理（付费代理）唯一的网站，给我这个问题是百思买，它平均需要10秒才能从我的请求得到回复

我在同一个数据中心的网站上测试了同一个代理，响应是即时的（如果需要，我可以发布日志）。更重要的是，当使用具有相同标头的我的住宅IP时，响应也是即时的。我唯一的结论是，百思买可能正在限制数据中心IP地址

但我不确定。 这是我的代码，下面是请求的日志

import requests
import logging
from http.client import HTTPConnection
from datetime import datetime
url = "https://www.bestbuy.ca/ecomm-api/availability/products?accept=application%2Fvnd.bestbuy.standardproduct.v1%2Bjson&accept-language=en-CA&locations=&postalCode=M5G2C3&skus=15078017|15166285|15084753|14954116|15038016|14953248|15147122|15229237|15053087|14950588|15309513|15318940|15178453|14953249|14953247|14954117|15309514|14967857|15000077|15000079|15000078|14966477|15441686|15317226|15324508|14953250|15201200|15053086|15053085|15309504|15309503"



log = logging.getLogger('urllib3')
log.setLevel(logging.DEBUG)
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
log.addHandler(ch)
HTTPConnection.debuglevel = 1


headers = {"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0"}


print(datetime.now())
r = requests.get(url,headers=headers)
print(datetime.now())

---

您从哪里获得用于此目的的代理？您是否尝试过使用另一个数据中心的代理服务器进行比较，或者甚至使用VPN（我知道这只是一个测试，速度会慢得多）

Proxy
2021-04-23 08:29:16.381893
Starting new HTTPS connection (1): www.bestbuy.ca:443
send: b'CONNECT www.bestbuy.ca:443 HTTP/1.0\r\n'
send: b'\r\n'
send: b'GET /ecomm-api/availability/products?accept=application%2Fvnd.bestbuy.standardproduct.v1%2Bjson&accept-language=en-CA&locations=&postalCode=M5G2C3&skus=15078017%7C15166285%7C15084753%7C14954116%7C15038016%7C14953248%7C15147122%7C15229237%7C15053087%7C14950588%7C15309513%7C15318940%7C15178453%7C14953249%7C14953247%7C14954117%7C15309514%7C14967857%7C15000077%7C15000079%7C15000078%7C14966477%7C15441686%7C15317226%7C15324508%7C14953250%7C15201200%7C15053086%7C15053085%7C15309504%7C15309503 HTTP/1.1\r\nHost: www.bestbuy.ca\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n'  
reply: 'HTTP/1.1 200 OK\r\n'
header: Cache-Control: no-cache, no-store
header: Pragma: no-cache
header: Content-Type: application/vnd.bestbuy.v1+json; charset=utf-8
header: Expires: -1
header: X-XSS-Protection: 1;mode=block
header: Content-Security-Policy: frame-ancestors http://*.bestbuy.com https://*.bestbuy.com http://*.bestbuy.ca https://*.bestbuy.ca
header: Strict-Transport-Security: max-age=604800
header: Access-Control-Allow-Origin: https://www.bestbuy.ca
header: Content-Length: 14966
header: Date: Fri, 23 Apr 2021 12:29:26 GMT
header: Connection: keep-alive
header: Set-Cookie: bm_sz=BBCBF57DAC1124BEB7E83D3CB1225CB1~YAAQLA3orHnlfPJ4AQAA1NO0/gt49j9YqjybC86gMX+V23AvV30C8q7y5B23PpNrYfb0kZyH4W6PBtibUrM2e3pSNrl+H6sX1KpzwAhXz8+hYfkIV93u/iGJWmWU5Bw7TjCKZusKpgwDy6FZN4dTrPmY9q4dG3g98K4qBSgWbntYpfeALZkXTsGy1DwG+VMB26Qjsu8RCVymy+IlQlVU7gxdIZbSwi3XaLpueashjSp0MXMVD+3qTWDkeH4LmGOwC83XFgADfa277Zrgojr7cu4GLHpZbCtSvXJevw==; Domain=.bestbuy.ca; Path=/; Expires=Fri, 23 Apr 2021 16:29:16 GMT; Max-Age=14390; HttpOnly
header: Set-Cookie: _abck=5BC0B07AFDBE7AF22FF07EC95D4836A5~-1~YAAQLA3orHrlfPJ4AQAA1NO0/gXRSlR2RRTQ0JyHuvqfX/YvJb/+3w+AbKdCLvW8oLPAlE3b28k/Ono96Y+fqM/7A3c0R+MB53Fqn37qRpdl4vNJk1GcijyhN6qpikuiFScf2wK0u6Fb4CoFUB5Q9sW7+70jC7Wo7ELEjkh9BtcUVzh1sZtU6ZpyTOBkiki4rtSC1SOh+0hzqoUZostRpcXzpbWn5ZCh59FMl4Vgqrk560sBy4MVLX5zwKfb8/5uOz6YlnL4Hdpwn4cqyCqVcPU5Inq7dw/V2vJpEKYIf4VIcoxekI8yxekUcU9k5zIhl0PGxEeICeu0lfk2/xd+ZeHxHB6MfBuLfqs8fRU918zSoCNLVWY2nW0IHiS4gpiR9n9K+KolJj0=~-1~-1~-1; Domain=.bestbuy.ca; Path=/; Expires=Sat, 23 Apr 2022 12:29:26 GMT; Max-Age=31536000; Secure
https://www.bestbuy.ca:443 "GET /ecomm-api/availability/products?accept=application%2Fvnd.bestbuy.standardproduct.v1%2Bjson&accept-language=en-CA&locations=&postalCode=M5G2C3&skus=15078017%7C15166285%7C15084753%7C14954116%7C15038016%7C14953248%7C15147122%7C15229237%7C15053087%7C14950588%7C15309513%7C15318940%7C15178453%7C14953249%7C14953247%7C14954117%7C15309514%7C14967857%7C15000077%7C15000079%7C15000078%7C14966477%7C15441686%7C15317226%7C15324508%7C14953250%7C15201200%7C15053086%7C15053085%7C15309504%7C15309503 HTTP/1.1" 200 14966
2021-04-23 08:29:26.915711

No Proxy
2021-04-23 08:29:59.060498
Starting new HTTPS connection (1): www.bestbuy.ca:443
send: b'GET /ecomm-api/availability/products?accept=application%2Fvnd.bestbuy.standardproduct.v1%2Bjson&accept-language=en-CA&locations=&postalCode=M5G2C3&skus=15078017%7C15166285%7C15084753%7C14954116%7C15038016%7C14953248%7C15147122%7C15229237%7C15053087%7C14950588%7C15309513%7C15318940%7C15178453%7C14953249%7C14953247%7C14954117%7C15309514%7C14967857%7C15000077%7C15000079%7C15000078%7C14966477%7C15441686%7C15317226%7C15324508%7C14953250%7C15201200%7C15053086%7C15053085%7C15309504%7C15309503 HTTP/1.1\r\nHost: www.bestbuy.ca\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n'  
reply: 'HTTP/1.1 200 OK\r\n'
header: Cache-Control: no-cache, no-store
header: Pragma: no-cache
header: Content-Type: application/vnd.bestbuy.v1+json; charset=utf-8
header: Expires: -1
header: X-XSS-Protection: 1;mode=block
header: Content-Security-Policy: frame-ancestors http://*.bestbuy.com https://*.bestbuy.com http://*.bestbuy.ca https://*.bestbuy.ca
header: Strict-Transport-Security: max-age=604800
header: Access-Control-Allow-Origin: https://www.bestbuy.ca
header: Content-Length: 14966
header: Date: Fri, 23 Apr 2021 12:29:59 GMT
header: Connection: keep-alive
header: Set-Cookie: bm_sz=212500C5B82925E3A19C63514C69D505~YAAQXQp8aN/UuOt4AQAAM1O1/gvf7rJUbnWym5Em4K9bx2wFzPjLJzto9qA7gmCGuYGzSY3A7YMlB5n7aQljNBB27Tfby36Xssf3RN0VxjUAehX6g5ely/YvqO03cWb93xFJAKDccQJy+8FSXR1FzzmGBCzA4m+QT/Rjz5yIaxRpIe9+69rmezQ2ktZE+rpqyPSvtgN1UFCYfwddSTxbRdv9/w9ArQHeTlFPy7YPD+cej9QKkhHc+x8lxwkSM2uqkHEtLgotMWYjnUmveB/TBtbNrICHPGnVpYU=; Domain=.bestbuy.ca; Path=/; Expires=Fri, 23 Apr 2021 16:29:59 GMT; Max-Age=14400; HttpOnly
header: Set-Cookie: _abck=28D63400F227C1762F13FCA16DFEEB54~-1~YAAQXQp8aODUuOt4AQAAM1O1/gWeUVLTk3EdaxAa9Nbok/1lfQfRK968DpMdhAUkayo4LkxHKoyhsB6bQJLOFDfvKbswh0SvpEDZBi0piE7l3NQwdzuh3Xk+u1aXDQliDEFhuGqhyjBNnQvfRxgVKZ7KUcHtbUwJD3IXnLqE72R1e4Tvmy3SAY0fgQG3g5Rmyj6nn1lyxKm1XMlAMe1HWAAK55hCHzSwSYgebPAQF6YIfPjbWtdLbi8H8B881i3dHZ2D4wwQYQ6wX7/ysVzfrVp0Z7pwBfSxIxpFkwxk2vMLOTexk0FuvDuexW0VsZX0TmpXG54dR89zF0ew2yGVBTz/DEcWvn8vcdb1ayWDPN16RkV1KL6VYud6yXBKtZpamLDjx/XFOK4=~-1~-1~-1; Domain=.bestbuy.ca; Path=/; Expires=Sat, 23 Apr 2022 12:29:59 GMT; Max-Age=31536000; Secure
https://www.bestbuy.ca:443 "GET /ecomm-api/availability/products?accept=application%2Fvnd.bestbuy.standardproduct.v1%2Bjson&accept-language=en-CA&locations=&postalCode=M5G2C3&skus=15078017%7C15166285%7C15084753%7C14954116%7C15038016%7C14953248%7C15147122%7C15229237%7C15053087%7C14950588%7C15309513%7C15318940%7C15178453%7C14953249%7C14953247%7C14954117%7C15309514%7C14967857%7C15000077%7C15000079%7C15000078%7C14966477%7C15441686%7C15317226%7C15324508%7C14953250%7C15201200%7C15053086%7C15053085%7C15309504%7C15309503 HTTP/1.1" 200 14966
2021-04-23 08:29:59.496942