Python gevent队列因LoopExit而失败
我想使用PythonGevent库实现一个生产者和多个消费者服务器。我的尝试是:Python gevent队列因LoopExit而失败,python,multithreading,queue,gevent,Python,Multithreading,Queue,Gevent,我想使用PythonGevent库实现一个生产者和多个消费者服务器。我的尝试是: class EmailValidationServer(): def __init__(self): self.queue = Queue() def worker(self): while True: json = self.queue.get() def handler(self,socket,address): fileobj = so
class EmailValidationServer():
def __init__(self):
self.queue = Queue()
def worker(self):
while True:
json = self.queue.get()
def handler(self,socket,address):
fileobj = socket.makefile()
content = fileobj.read(max_read)
contents = json.loads(content)
for content in contents:
self.queue.put(content)
def daemon(self,addr='127.0.0.1',num_thread=5):
pool = Pool(1000)
server = StreamServer((addr, 6000),self.handler,spawn=pool) # run
pool = ThreadPool(num_thread)
for _ in range(num_thread):
pool.spawn(self.worker)
server.serve_forever()
if __name__ == "__main__":
email_server = EmailValidationServer()
email_server.daemon()
我使用了gevent.queue.queue中的队列。它向我提供了错误信息:
LoopExit: This operation would block forever
(<ThreadPool at 0x7f08c80eef50 0/4/5>,
<bound method EmailValidationServer.worker of <__main__.EmailValidationServer instance at 0x7f08c8dcd998>>) failed with LoopExit
LoopExit:此操作将永远阻塞
(,
)使用LoopExit失败
问题:但当我将队列从gevent的实现更改为python内置库时,它就可以工作了。我不知道原因,我想支持它们的实现之间存在差异。我不知道为什么gevent不允许无限等待。有人能解释一下吗?感谢advance我建议您可以使用
gevent.queue.JoinableQueue()
而不是Python内置的queue()
。您可以参考官方队列指南了解API用法()
如果您再次遇到异常,您最好充分了解Gevent corouinte控制流的原理……一旦您了解了要点,那就没什么大不了的了。:) @SuperBiasedMan thans用于格式化我的代码块:)
def worker():
while True:
item = q.get()
try:
do_work(item)
finally:
q.task_done()
q = JoinableQueue()
for i in range(num_worker_threads):
gevent.spawn(worker)
for item in source():
q.put(item)
q.join() # block until all tasks are done