Python 无授权重定向
根据标准,它应该简单到:Python 无授权重定向,python,amazon-s3,python-requests,bitbucket,urllib,Python,Amazon S3,Python Requests,Bitbucket,Urllib,根据标准,它应该简单到: data = self.http_pool.urlopen('GET', file_url, preload_content=False, retries=max_download_retries) request.add_unredirected_header(key, header) Add a header that will not b
data = self.http_pool.urlopen('GET', file_url,
preload_content=False,
retries=max_download_retries)
request.add_unredirected_header(key, header)
Add a header that will not be added to a redirected request.
但我似乎找不到任何关于如何实现这一目标的例子
我正在使用pyupdater从bitbucket下载更新并启动最新版本的exe。我正在使用此库创建一个连接到bitbucket fine的脚本,但随后它会使用nauthorization:Basic\r\n\r\n
(这是bitbucket auth)重定向到Amazon,这意味着我得到了“HTTP/1.1400错误请求\r\n”
。Amazon不支持基本身份验证。这应该很容易解决,但我在这个问题上找不到太多
提供的解决方案需要手动重新创建每个重定向请求。如果我必须为我上传的新文件这样做的话,这将成为一个不断增长的列表,并且很快变得单调乏味。它也不会继续脚本的其余部分,而是下载到同一目录
这就是Pyupdater处理下载的方式,这可能是解决问题的地方
downloader.py的第366行:
data = self.http_pool.urlopen('GET', file_url,
preload_content=False,
retries=max_download_retries)
任何关于如何修复此问题,使其不再产生此错误的想法
完全错误(ctrl f->400):
Python main.py
调试:根目录:版本-2.5.1
调试:pyupdater。客户端:pyupdater版本2.5.1
目前的版本是1.3
{'authorization':'Basic'}
调试:pyupdater.client:正在设置目录。。。
调试:pyupdater.client:下载密钥文件
调试:pyupdater.client.downloader:请求的Url:https://api.bitbucket.org/2.0/repositories/ brofewfewewewef/eee/downloads/keys.gz
调试:urllib3.connectionpool:启动新的HTTPS连接(1):api.bitbucket.org
发送:b'GET/2.0/repositories/brofewfefwewewewef/eee/downloads/keys.gz HTTP/1.1\r\n主机:api.bitbucket.org\r\n接受编码:标识\r\n授权:基本\r\n\r\n
答复:“找到HTTP/1.1 302\r\n”
调试:urllib3.connectionpool:https://api.bitbucket.org:443 “GET/2.0/repositories/brofewfewewewef/eee/downloads/keys.gz HTTP/1.1”3020
调试:urllib3.util.retry:递增的重试次数(url=)https://api.bitbucket.org/2.0/repositories/brofewfefwefewef/eee/downloads/keys.gz“):重试(总计=2,连接=无,读取=无,重定向=无,状态=无)
信息:urllib3.poolmanager:重定向https://api.bitbucket.org/2.0/repositories/ brofewfewewewef/eee/downloads/keys.gz->https://bbuseruploads.s3.amazonaws.com/a0e395b6-0c54-4efb-9074-57ec4190020b/downloads/3fc0be6d-ca69-42d3-9711-fbb5cfd2bc38/keys.gz?Signature=&Expires=1515976464&AWSAccessKeyId=&versionId=n.ymY11KRkq36Xozy25aChvfUT.YzTf5&response-内容处置=附件%3B%20filename%3D%22keys.gz%22
调试:urllib3.connectionpool:启动新的HTTPS连接(1):bbuseruploads.s3.amazonaws.com
header:Server header:Vary header:Content-Type header:X-OAuth-Scopes header:Strict Transport-Security header:Date header:Location header:X-Served-By header:ETag header:X-Static-Version header:X-Content-Type-Options header:X-Accepted-OAuth-Scopes header:X-Credential-Type header:X-Render-Time header:Connection header:X-Request-Count标题:X-Frame-Options标题:X-Version标题:内容长度发送:b'GET/a0e395b6-0c54-4efb-9074-57ec4190020b/downloads/3fc0be6d-ca69-42d3-9711-fbb5cfd2bc38/keys.gz?签名=&Expires=1515976464&AWSAccessKeyId=&versionId=n.ymy11krkq36xozy25achvput.yzf5&response内容处置=附件%3B%20filename%3D%22keys.gz%22 HTTP/1.1\r\n主机:bbuseruploads.s3.amazonaws.com\r\n接受编码:标识\r\n授权:基本\r\n\r\n'
答复:“HTTP/1.1 400错误请求\r\n”
调试:urllib3.connectionpool:https://bbuseruploads.s3.amazonaws.com:443 “GET/a0e395b6-0c54-4efb-9074-57ec4190020b/downloads/3fc0be6d-ca69-42d3-9711-fbb5cfd2bc38/keys.gz?签名=&Expires=1515976464&AWSAccessKeyId=&versionId=n.ymy11krkq36xozy25achvput.YzTf5&response内容处置=附件%3B%20filename%3D%22keys.gz%22 HTTP/1.1”400无
调试:pyupdater.client.downloader:资源URL:https://api.bitbucket.org/2.0/repositories/brofewfefwefewef/eee/downloads/keys.gz
调试:pyupdater.client.downloader:获取的内容长度为:无
调试:pyupdater.client.downloader:内容长度不在标题中
调试:pyupdater.client.downloader:回调不会显示剩余时间或下载百分比。
调试:pyupdater.client.downloader:使用文件作为存储,因为文件太大
调试:pyupdater.client.downloader:块大小:1036
调试:pyupdater.client.downloader:{'total':无,'downloaded':519,'status':'downloading','percent_complete':'-.-%,'time':'-:'-:'}
调试:pyupdater.client.downloader:{'total':无,'downloaded':519,'status':'finished','percent_complete':'-.-%','time':'00:00'}
调试:pyupdater.client.downloader:下载完成
调试:pyupdater.client.downloader:没有要验证的哈希
警告:pyupdater.client.downloader:下载的文件非常大,将其读入内存可能会使应用程序崩溃
调试:pyupdater。客户端:未能解压缩gzip文件
调试:pyupdater。客户端:版本文件下载失败
标头:x-amz-request-id标头:x-amz-id-2标头:内容类型标头:传输编码标头:日期标头:连接标头:服务器{'authorization':'Basic'}
调试:pyupdater.client:不是gzip文件(我刚刚检查过,我认为这是pyupdater的问题)(我不知道它是什么,从未使用过)
它似乎假设所有响应的主体都将在GZIP中被压缩。我找不到任何标志可以阻止这种假设。实际内容实际上根本没有被压缩
以下是pyupdater中的一些相关代码:
pyupdater/client/\uuuu init\uuuu.py:
def _get_manifest_from_http(self):
log.debug('Downloading online version file')
try:
fd = _FD(self.version_file, self.update_urls, verify=self.verify,
urllb3_headers=self.urllib3_headers)
data = fd.download_verify_return()
try:
import ipdb
ipdb.set_trace()
decompressed_data = _gzip_decompress(data)
except IOError:
log.debug('Failed to decompress gzip file')
# Will be caught down below.
# Just logging the error
raise
log.debug('Version file download successful')
# Writing version file to application data directory
self._write_manifest_2_filesystem(decompressed_data)
return decompressed_data
except Exception as err:
log.debug('Version file download failed')
log.debug(err, exc_info=True)
return None
以下是我收到的数据示例:
ipdb> data
b'{"type": "error", "error": {"message": "keys.gz"}}'
我认为你应该在上开一张罚单,看看他们是否能进一步帮助你。Edit-1:
您需要为main.py
使用以下代码,而不需要对downloader.py
from __future__ import print_function
import urllib3.poolmanager
orig_urlopen = urllib3.poolmanager.PoolManager.urlopen
def new_urlopen(self, method, url, redirect=True, **kw):
if "s3.amazonaws.com" in url and 'authorization' in self.headers:
self.headers.pop('authorization')
return orig_urlopen(self, method, url, redirect, **kw)
urllib3.poolmanager.PoolManager.urlopen = new_urlopen
import logging
from selenium import webdriver
logging.basicConfig(level=logging.DEBUG)
from client_config import ClientConfig
from pyupdater.client import Client, AppUpdate
import http.client as http_client
http_client.HTTPConnection.debuglevel = 1
def check_for_update():
client = Client(ClientConfig(), refresh=True, headers={'basic_auth': '<username>:<password>'})
app_update = client.update_check(ClientConfig.APP_NAME, ClientConfig.APP_VERSION, channel='stable')
if app_update is not None:
if app_update.download():
if isinstance(app_update, AppUpdate):
app_update.extract_restart()
return True
else:
app_update.extract()
return True
return False
def main():
print('Current version is ', ClientConfig.APP_VERSION)
if check_for_update():
print('there\'s a new update :D')
# driver = webdriver.Firefox()
# driver.get('http://stackoverflow.com')
if __name__ == "__main__":
main()
使用上面的补丁进行了一次示例测试
import urllib3
pool = urllib3.PoolManager()
pool.headers.update({'Authorization': 'Basic XYZ=='})
r = pool.urlopen('GET', 'https://api.bitbucket.org/2.0/repositories/brofewfefwefewef/eee/downloads/keys.gz')
print(r.data)
您需要在导入pyupdater之前执行代码,pyupdater是一个非常棒的库,不要浪费时间做其他事情,除非
import urllib3.poolmanager
orig_urlopen = urllib3.poolmanager.PoolManager.urlopen
def new_urlopen(self, method, url, redirect=True, **kw):
if "s3.amazonaws.com" in url and 'Authorization' in self.headers:
self.headers.pop('Authorization')
return orig_urlopen(self, method, url, redirect, **kw)
urllib3.poolmanager.PoolManager.urlopen = new_urlopen
import urllib3
pool = urllib3.PoolManager()
pool.headers.update({'Authorization': 'Basic XYZ=='})
r = pool.urlopen('GET', 'https://api.bitbucket.org/2.0/repositories/brofewfefwefewef/eee/downloads/keys.gz')
print(r.data)
import requests
import zlib
def download(url, username, password):
r = requests.get(url, auth=requests.auth.HTTPBasicAuth(username, password))
r.raise_for_status()
return zlib.decompress(r.content, 15 + 32)
download('https://api.bitbucket.org/2.0/repositories/brofewfefwefewef/eee/downloads/keys.gz', 'brofewfefwefewef', your_password)