ipython 0.13 zmq错误

ipython 0.13 zmq错误,ipython,pyzmq,ipython-parallel,Ipython,Pyzmq,Ipython Parallel,我遇到了ipython集群的奇怪行为。计算完成,但许多结果从未送达客户(发动机在完成第一次计算后只是怠速运转) 我怀疑zmq有问题,因为1)我不时看到以下错误: File "/data/misc/nano/python/env_stable/lib/python2.7/site-packages/IPython/parallel/client/asyncresult.py", line 118, in get if not self.ready(): File "/data/mi

我遇到了ipython集群的奇怪行为。计算完成,但许多结果从未送达客户(发动机在完成第一次计算后只是怠速运转)

我怀疑zmq有问题,因为1)我不时看到以下错误:

  File "/data/misc/nano/python/env_stable/lib/python2.7/site-packages/IPython/parallel/client/asyncresult.py", line 118, in get
    if not self.ready():
  File "/data/misc/nano/python/env_stable/lib/python2.7/site-packages/IPython/parallel/client/asyncresult.py", line 132, in ready
    self.wait(0)
  File "/data/misc/nano/python/env_stable/lib/python2.7/site-packages/IPython/parallel/client/asyncresult.py", line 142, in wait
    self._ready = self._client.wait(self.msg_ids, timeout)
  File "/data/misc/nano/python/env_stable/lib/python2.7/site-packages/IPython/parallel/client/client.py", line 1058, in wait
    self.spin()
  File "/data/misc/nano/python/env_stable/lib/python2.7/site-packages/IPython/parallel/client/client.py", line 1015, in spin
    self._flush_results(self._task_socket)
  File "/data/misc/nano/python/env_stable/lib/python2.7/site-packages/IPython/parallel/client/client.py", line 814, in _flush_results
    idents,msg = self.session.recv(sock, mode=zmq.NOBLOCK)
  File "/data/misc/nano/python/env_stable/lib/python2.7/site-packages/IPython/zmq/session.py", line 642, in recv
    idents, msg_list = self.feed_identities(msg_list, copy)
  File "/data/misc/nano/python/env_stable/lib/python2.7/site-packages/IPython/zmq/session.py", line 673, in feed_identities
    idx = msg_list.index(DELIM)
ValueError: '<IDS|MSG>' is not in list

Additionally IPython.zmq has two test failures:

======================================================================
ERROR: test_send (IPython.zmq.tests.test_session.TestSession)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/clusterdata/python/env_stable/lib/python2.7/site-packages/IPython/zmq/tests/test_session.py", line 76, in test_send
    socket = MockSocket(zmq.Context.instance(),zmq.PAIR)
  File "/clusterdata/python/env_stable/lib/python2.7/site-packages/IPython/zmq/tests/test_session.py", line 34, in __init__
    self.data = []
  File "/clusterdata/python/env_stable/lib/python2.7/site-packages/zmq/sugar/attrsettr.py", line 38, in __setattr__
    self.__class__.__name__, upper_key)
AttributeError: MockSocket has no such option: DATA

======================================================================
ERROR: test_send (IPython.zmq.tests.test_session.TestSession)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/clusterdata/python/env_stable/lib/python2.7/site-packages/zmq/tests/__init__.py", line 108, in tearDown
    raise RuntimeError("context could not terminate, open sockets likely remain in test")
RuntimeError: context could not terminate, open sockets likely remain in test

----------------------------------------------------------------------
文件“/data/misc/nano/python/env_stable/lib/python2.7/site packages/IPython/parallel/client/asyncresult.py”,get中的第118行
如果不是self.ready():
文件“/data/misc/nano/python/env_stable/lib/python2.7/site packages/IPython/parallel/client/asyncresult.py”,第132行,准备就绪
self.wait(0)
文件“/data/misc/nano/python/env_stable/lib/python2.7/site packages/IPython/parallel/client/asyncresult.py”,第142行,等待中
self.\u ready=self.\u client.wait(self.msg\u id,超时)
文件“/data/misc/nano/python/env_stable/lib/python2.7/site packages/IPython/parallel/client/client.py”,第1058行,正在等待
self.spin()
文件“/data/misc/nano/python/env_stable/lib/python2.7/site packages/IPython/parallel/client/client.py”,第1015行,spin格式
self.\u刷新\u结果(self.\u任务\u套接字)
文件“/data/misc/nano/python/env_stable/lib/python2.7/site packages/IPython/parallel/client/client.py”,第814行,在“刷新”结果中
idents,msg=self.session.recv(sock,mode=zmq.NOBLOCK)
recv中的文件“/data/misc/nano/python/env_stable/lib/python2.7/site packages/IPython/zmq/session.py”,第642行
idents,msg_list=self.feed_标识(msg_list,copy)
文件“/data/misc/nano/python/env_stable/lib/python2.7/site packages/IPython/zmq/session.py”,第673行,在提要中
idx=msg_list.index(DELIM)
ValueError:“”不在列表中
此外,IPython.zmq还有两个测试失败:
======================================================================
错误:test_send(IPython.zmq.tests.test_session.TestSession)
----------------------------------------------------------------------
回溯(最近一次呼叫最后一次):
文件“/clusterdata/python/env_stable/lib/python2.7/site packages/IPython/zmq/tests/test_session.py”,第76行,在test_send中
socket=MockSocket(zmq.Context.instance(),zmq.PAIR)
文件“/clusterdata/python/env_stable/lib/python2.7/site packages/IPython/zmq/tests/test_session.py”,第34行,在__
self.data=[]
文件“/clusterdata/python/env_stable/lib/python2.7/site packages/zmq/sugar/attrstr.py”,第38行,在__
self.\uuuuu类\uuuuuu.\uuuuuuu名称\uuuuuuu,上键)
AttributeError:MockSocket没有这样的选项:DATA
======================================================================
错误:test_send(IPython.zmq.tests.test_session.TestSession)
----------------------------------------------------------------------
回溯(最近一次呼叫最后一次):
文件“/clusterdata/python/env_stable/lib/python2.7/site packages/zmq/tests/_init__uuu.py”,第108行,分拆
raise RUNTIMERROR(“上下文无法终止,打开的套接字可能仍在测试中”)
运行时错误:上下文无法终止,打开的套接字可能仍在测试中
----------------------------------------------------------------------
我使用pyzmq 13.0.0(由pip安装)和zeromq 3.2.2,由pyzmq的安装程序编译。我使用ipython13.1和python2.7.3

有没有关于这可能是什么的建议,如果没有的话,我如何能够找出这些错误发生的更多信息


更新:事实证明,速度放缓是由于ipcontroller的任务队列很长,然后占用了100%的CPU,严重滞后。这是一个单独的问题,但我仍然希望得到上述反馈。

由@minrk在评论中回答。ZMQ错误并不重要,性能是由调度引起的,通过设置
TaskScheduler.hwm=0

可以解决。MockSocket错误只影响测试本身,并在0.13.2中修复。知道其他错误是什么吗?同样根据更新,ipcontroller是否应该像地狱一样延迟队列中的4000个作业(如果它没有延迟几百个作业的话)?显然不应该,但这并不意味着您的系统有问题。如果您有很多作业,我强烈建议将TaskScheduler.hwm设置为较大的值。实际上,任何较大的值都没有帮助,但0起作用。