Python FutureSession:套接字连接未关闭

我正在做一个PyQt程序，它主要从互联网上收集数据。在本例中，我试图从RSS网页获取数据

假设self.feed是包含所有文章的RSS页面，假设“entry”是一篇文章。“entry.url”是网站上文章的原始页面

from requests_futures.sessions import FuturesSession

self.session_pages = FuturesSession(max_workers=20)
for entry in self.feed.entries:
    future = self.session_pages.get(entry.url, timeout=10)
    future.add_done_callback(my_call_back)

基本上我就是这样做的。它嵌入在一个PyQt线程中，我同时运行多个线程，但我认为问题不是来自PyQt

我的问题是，我认为未来不会关闭连接，即使它们已经完成。我这样检查：

lsof -i | grep "python" | wc -l

losf-i提供连接中涉及的打开文件。命令的其余部分是计算打开的文件数。 这个数字没有停止增长（大约900），然后我得到以下错误：

(python:28285): GLib-ERROR **: Creating pipes for GWakeup: Too many open files
[1]    28285 trace trap (core dumped)  python gui.py

我认为问题来自未来，但实际上我不确定

我试过这样的方法：

self.session_pages.shutdown()

在线程的末尾，但它不起作用

---

你有什么想法吗？

我没有看到python的

concurrent.futures

中的

FutureSession

？我在这里做一些假设

除非回调对每个

self.session\u页面都是唯一的。get（…）

我认为

future.add\u done\u回调（my\u call\u back）

行可能正在创建新的，并覆盖回调的对象id，或者可能不正确

在这里，我可以在您使用的内容中找到对

FutureSession

的参考：

from pprint import pprint
from requests_futures.sessions import FuturesSession

session = FuturesSession()

def bg_cb(sess, resp):
    # parse the json storing the result on the response object
    resp.data = resp.json()

future = session.get('http://httpbin.org/get', background_callback=bg_cb)
# do some other stuff, send some more requests while this one works
response = future.result()
print('response status {0}'.format(response.status_code))
# data will have been attached to the response object in the background
pprint(response.data)

尝试设置

background\u回调

更新：

• 唯一被覆盖/包装的是as和

  并发。futures

  既没有get，也没有request

我会尝试使用

self.session\u页面.request

而不是

self.session\u页面.get

，因为

FutureSession

由线程池执行器和

请求.Sessions

是的，情况就是这样：

(Pdb) inspect.getmro(FuturesSession)
(<class '__main__.FuturesSession'>, <class 'requests.sessions.Session'>, <class 'requests.sessions.SessionRedirectMixin'>, <class 'object'>)
(Pdb) vars()
{'DEFAULT_POOLSIZE': 10, '__return__': None, '__spec__': None, 'inspect': <module 'inspect' from '/usr/lib/python3.4/inspect.py'>, '__file__': 'requestsfutures.py', 'FuturesSession': <class '__main__.FuturesSession'>, 'HTTPAdapter': <class 'requests.adapters.HTTPAdapter'>, 'ThreadPoolExecutor': <class 'concurrent.futures.thread.ThreadPoolExecutor'>, 'Session': <class 'requests.sessions.Session'>, '__name__': '__main__', '__cached__': None, '__doc__': "\nrequests_futures\n~~~~~~~~~~~~~~~~\n\nThis module provides a small add-on for the requests http library. It makes use\nof python 3.3's concurrent.futures or the futures backport for previous\nreleases of python.\n\n    from requests_futures import FuturesSession\n\n    session = FuturesSession()\n    # request is run in the background\n    future = session.get('http://httpbin.org/get')\n    # ... do other stuff ...\n    # wait for the request to complete, if it hasn't already\n    response = future.result()\n    print('response status: {0}'.format(response.status_code))\n    print(response.content)\n\n", 'pdb': <module 'pdb' from '/usr/lib/python3.4/pdb.py'>, '__loader__': <_frozen_importlib.SourceFileLoader object at 0x7f6d84194470>, '__builtins__': <module 'builtins' (built-in)>, '__package__': None}
(Pdb) vars().keys()
dict_keys(['DEFAULT_POOLSIZE', '__return__', '__spec__', 'inspect', '__file__', 'FuturesSession', 'HTTPAdapter', 'ThreadPoolExecutor', 'Session', '__name__', '__cached__', '__doc__', 'pdb', '__loader__', '__builtins__', '__package__'])
(Pdb) vars()['FuturesSession']
<class '__main__.FuturesSession'>
(Pdb) vars()['FuturesSession'].get
<function Session.get at 0x7f6d80c07488>
(Pdb) vars()['Session'].get
<function Session.get at 0x7f6d80c07488>

（Pdb）检查.getmro（未来会话）
(, , )
（Pdb）vars（）
{'DEFAULT_POOLSIZE'：10'，'uuu return_uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu“\n请求\u futures\n~~~~~~~~~~~~~~~~\n\n此模块为请求http库提供一个小的附加组件。它使用python 3.3的concurrent.futures或python以前\n次发布的futures backport。\n\n来自请求\u futures import futuresession\n\n session=futuresession（）\n#请求在后台运行\n future=session.get（'http://httpbin.org/get“）\n#…执行其他操作…\n#如果请求尚未完成，请等待请求完成\n response=future.result（）\n print（'response status:{0}'。format（response.status_code））\n print（response.content）\n\n“，‘pdb’：，‘加载程序’：，‘内置程序’：，‘包’：无}”
（Pdb）vars（）.keys（）
dict_键（['DEFAULT_POOLSIZE'、'uuu return_uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu
（Pdb）vars（）['FutureSession']
（Pdb）vars（）['FutureSession']。获取
（Pdb）vars（）['Session'].get

好的，你是rigt@jm。 get（）调用只是对requests.get的调用。所以我用了这里的答案：

更具体地说：

future = self.session_pages.get(url, timeout=20, headers={'Connection':'close'})

---

现在lsof表示正常数字。谢谢。

是的，你说得对，我的错。我编辑了我的问题并添加了导入。但实际上，FutureSession是ThreadPoolExecutor上的一个包装器（我想是这个类，我不记得了），FutureSession的未来基本上是并发的。我认为通过调用

get

调用使用的是常规的

请求

get方法，根本没有调用任何重写的回调/方法。很高兴我能让你继续做下去。如果你还记得的话，在运行

lsof-I | grep“python”| wc-l

之后你得到了什么结果？