单会话多个python请求中的post/get

单会话多个python请求中的post/get,python,web-crawler,python-requests,Python,Web Crawler,Python Requests,我正在尝试编写一个爬虫程序,使用python请求模块自动下载一些文件。然而,我遇到了一个问题 我初始化了一个新的请求会话,然后使用post方法登录网站,之后只要我尝试使用post/get方法(下面是一个简化代码): 它将报告如下所示的错误: Traceback (most recent call last): File"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/r

我正在尝试编写一个爬虫程序,使用python请求模块自动下载一些文件。然而,我遇到了一个问题

我初始化了一个新的请求会话,然后使用post方法登录网站,之后只要我尝试使用post/get方法(下面是一个简化代码):

它将报告如下所示的错误:

Traceback (most recent call last):
File"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 372, in _make_request
httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 374, in _make_request
httplib_response = conn.getresponse()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1162, in getresponse
raise ResponseNotReady(self.__state)
http.client.ResponseNotReady: Request-sent

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/adapters.py", line 370, in send
timeout=timeout
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 597, in urlopen
_stacktrace=sys.exc_info()[2])
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/util/retry.py", line 245, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/packages/six.py", line 309, in reraise
raise value.with_traceback(tb)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 374, in _make_request
httplib_response = conn.getresponse()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1162, in getresponse
raise ResponseNotReady(self.__state)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', ResponseNotReady('Request-sent',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "test.py", line 280, in <module>
test()
File "test.py", line 273, in test
emuch1.getEbook()
File "test.py", line 146, in getEbook
self.downloadEbook(ebook)
File "test.py", line 179, in downloadEbook
file_url=self.downloadEbookGetFileUrl(ebook).decode('gbk')
File "test.py", line 211, in downloadEbookGetFileUrl
download_url=self.downloadEbookGetUrl(ebook)
File "test.py", line 200, in downloadEbookGetUrl
respond_ebook=self.session.get(ebook_url)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 477, in get
return self.request('GET', url, **kwargs)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/adapters.py", line 415, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ResponseNotReady('Request-sent',))
回溯(最近一次呼叫最后一次):
文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site packages/requests/packages/urllib3/connectionpool.py”,第372行,在请求中
httplib_response=conn.getresponse(缓冲=True)
TypeError:getresponse()获得意外的关键字参数“buffering”
在处理上述异常期间,发生了另一个异常:
回溯(最近一次呼叫最后一次):
文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site packages/requests/packages/urllib3/connectionpool.py”,urlopen第544行
正文=正文,标题=标题)
文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site packages/requests/packages/urllib3/connectionpool.py”,第374行,在请求中
httplib_response=conn.getresponse()
getresponse中的文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py”,第1162行
提升响应状态(自身状态)
http.client.responseNodeTready:已发送请求
在处理上述异常期间,发生了另一个异常:
回溯(最近一次呼叫最后一次):
文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site packages/requests/adapters.py”,第370行,在send中
超时=超时
文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site packages/requests/packages/urllib3/connectionpool.py”,urlopen第597行
_stacktrace=sys.exc_info()[2])
文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site packages/requests/packages/urllib3/util/retry.py”,第245行,增量
升起六个。重新升起(类型(错误),错误,_stacktrace)
文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/packages/six.py”,第309行,重新登录
通过_回溯(tb)提升值
文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site packages/requests/packages/urllib3/connectionpool.py”,urlopen第544行
正文=正文,标题=标题)
文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site packages/requests/packages/urllib3/connectionpool.py”,第374行,在请求中
httplib_response=conn.getresponse()
getresponse中的文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py”,第1162行
提升响应状态(自身状态)
requests.packages.urllib3.exceptions.ProtocolError:(“连接已中止”)、ResponseNodeTready(“请求已发送”))
在处理上述异常期间,发生了另一个异常:
回溯(最近一次呼叫最后一次):
文件“test.py”,第280行,在
测试()
文件“test.py”,第273行,在测试中
emuch1.getEbook()
getEbook中第146行的文件“test.py”
下载电子书(电子书)
下载电子书中的文件“test.py”,第179行
file_url=self.downloadEbookGetFileUrl(电子书).decode('gbk'))
下载电子书GetFileURL中的文件“test.py”,第211行
download_url=self.downloadEbookGetUrl(电子书)
文件“test.py”,第200行,在downloadEbookGetUrl中
respond\u ebook=self.session.get(ebook\u url)
get中的文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site packages/requests/sessions.py”,第477行
返回self.request('GET',url,**kwargs)
文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site packages/requests/sessions.py”,请求中第465行
resp=自我发送(准备,**发送)
文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site packages/requests/sessions.py”,第573行,在send中
r=适配器.send(请求,**kwargs)
文件“/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site packages/requests/adapters.py”,第415行,在send中
raise CONNECTIONERR(错误,请求=请求)
requests.exceptions.ConnectionError:(“连接已中止”)、ResponseNodeTready(“请求已发送”))

我完全不知道为什么会这样,有人能帮我吗

请求使用urllib3的内部版本。我的印象是,内部urllib3和请求本身之间存在版本不匹配

httplib_response=conn.getresponse(缓冲=True)类型错误: getresponse()获得意外的关键字参数“buffering”

似乎表明请求正在调用urllib3(内部版本,而不是Python),但希望指定不存在的“缓冲”

其他问题与我所经历的相似

最新的requests版本(2.6.*)还有一些其他问题正在解决。我怀疑你正在使用那个版本。尝试回到以前的版本2.4.1,甚至2.2.1。如果在程序顶部指定要使用的版本,则可以保留安装的最新版本:

__requires__ = ["requests==2.2.1"]
import pkg_resources
(至少在导入请求本身之前)


解决方案:上周我与开发团队交换了几封邮件,看起来他们很快就完成了2.7版本的修复!(事实上,我看到它是昨天才上传的)。因此,如果您遇到类似问题,请下载最新版本

通过将请求升级到最新版本,问题已经解决,可能是2.6中的错误(可能是这个版本,不太确定)

多谢各位!我已经升级到2.7了,问题已经解决了!但我是Python新手,对您的答案非常感兴趣。您提到,通过“requires=[“requests==2.2.1”]import pkg_resources”,回到上一个版本,pkg_resources是什么?你是说我应该把这两行写在
__requires__ = ["requests==2.2.1"]
import pkg_resources