如何使用Pyramid优雅地处理python子流程

我将一些伪代码放在一起，启动一个进程（exist_view），该进程调用一个shell脚本，然后循环通过一个状态页面，定期检查该进程是否生成了返回代码，然后呈现结果

但是，我得到的结果是，子流程在客户端上锁定呈现，直到完成为止，这使得“客户端的定期状态更新”没有意义。我真的需要能够向客户机显示事情正在进行，因为finally项目将运行几个连续的子流程，其中一些可能需要几分钟（这是针对自定义设备，而不是某些网站）。下面是代码片段。有什么建议吗

请注意，我使用的是Python 2.6.5和Pyramid 1.4

@view_config(route_name='waiting', renderer='waiting.mako')
def waiting_view(request, process):
    while process.poll() == None:
        time.sleep(1)
    print str(process.returncode)
    return {}

@view_config(route_name='exist', renderer='exist.mako')
def exist_view(request):
    process = subprocess.Popen(['bash', '-c', './dummy.sh'], stdout=subprocess.PIPE)
    return HTTPFound(request.route_url('waiting', process=process))

---

我的幻想在#金字塔IRC频道中被驱散

tl；dr表示有3种选择：

分叉进程而不是使用子进程

使用队列引擎（如芹菜）

使用生成器来解析输入的结果

以下是聊天日志，以便进一步说明：

<phira> hewhocutsdown: that's never going to work
<phira> hewhocutsdown: I can see what you're trying to do, but the threading/request system doesn't work like that
* chrisrossi has quit (Ping timeout: 264 seconds)
<hewhocutsdown> hrm... is there another approach I could take, or is that strictly precluded?
<phira> hewhocutsdown: what you want is most likely to use one of the queue engines like celery or gearman to run the process from a pool
<phira> that approach is probably the most normal production approach to take
<phira> if you're after a simpler, quicker hack you have two options. The first is to fork the whole process and have it write to a file or something, that waiting_view can read from to get status
<phira> the second is to use a generator as the return on exist_view and let it feed out the results of the process bit by bit as they arrive, effectively turning it into a long poll
<hewhocutsdown> yeah, I looked at Celery but I don't have enough server control to be able to load it. // the forking idea could be done // what do you mean by a generator?
<hewhocutsdown> and to clarify; the .sh is basically copying and configuring Xen images
<hewhocutsdown> so they're simple scripts but they may take a while to respond
<pdobrogost> graffic: it doesn't need to be stable to be used in trasactions. what you see (or you image you see) is irrelevant here
<phira> ok, in that case I'd go with the forking thing
<phira> the one problem you're gonna have is that you can lose track of the processes (ie you fork one, and forget its there and it zombies or similar), you can deal with this in a variety of ways using process pools but they're not really a pyramid issue
<phira> the key point to remember is that you can't set a variable in one view, and have another view do things with it. Your views are (often) in separate thread contexts and unless you play a very careful game of thread safety they can't talk to each other like that, any info they want to share needs to go through the database or filesystem.
<hewhocutsdown> yeah, I don't have a db, at present it's just filesystem (calling shell scripts/reading return codes/writing text files). I'll take a stab at prototyping the fork method. Just so I know, how would the generator method work?
<phira> with the generator you return aresponse that basically returns the lines as they arrive from the script or similar, slowly (so the request takes ages, basically, but starts feeding out information almost immediately). You can then use xhr on the client side to call that url (once) to start the process and read the results as they happen
<phira> the risk of doing so is that you have no way of restarting it. If the client loses the connection, they cannot re-poll the endpoint because that will start a new process, instead of talk to the old one
<hewhocutsdown> got it. yes, I did something similar before I switched to using return codes.
<phira> there are strategies to get around that, but it's rarely worth the effort.
<hewhocutsdown> alright, I'll look into forking and process pooling.
<hewhocutsdown> thank you
<phira> no problem
<phira> also for what it's worth, this is probably the right place for almost any pyramid related question, the people here know all the things and they're usually around. THat's not always the case for some channels so I figure it's worth saying.

hewhocutsdown:那是行不通的
hewhocutsdown：我可以看到您正在尝试做什么，但是线程/请求系统不是这样工作的
*chrisrossi已退出（Ping超时：264秒）
人力资源管理。。。我是否可以采取另一种方法，或者严格排除这种方法？
hewhocutsdown：您最希望的是使用芹菜或gearman之类的队列引擎从池中运行流程
这种方法可能是最正常的生产方法
如果你想要一个更简单，更快的黑客，你有两个选择。第一种方法是将整个过程分叉，并将其写入一个文件或其他东西，等待视图可以从中读取以获取状态
第二种方法是使用生成器作为exist_视图的返回，让它在进程结果到达时一点一点地输出结果，从而有效地将其转化为一个长轮询
是的，我看过芹菜，但我没有足够的服务器控制来加载它分叉的想法可以实现//你说的发电机是什么意思？
并澄清,；sh基本上是复制和配置Xen映像
所以它们是简单的脚本，但可能需要一段时间才能响应
涂鸦：它不需要是稳定的，以用于运输。你所看到的（或你所看到的图像）在这里是无关紧要的
好吧，那样的话，我会用叉子
您将遇到的一个问题是，您可能会失去对流程的跟踪（即，您分叉了一个，忘记了它的存在和它的僵尸或类似），您可以使用流程池以多种方式处理这一问题，但它们实际上不是金字塔问题
要记住的关键一点是，不能在一个视图中设置变量，而让另一个视图来处理它。您的视图（通常）处于不同的线程上下文中，除非您玩一个非常谨慎的线程安全游戏，否则它们不能像那样相互交谈，它们想要共享的任何信息都需要通过数据库或文件系统。
是的，我没有db，目前它只是文件系统（调用shell脚本/读取返回代码/编写文本文件）。我将尝试一下fork方法的原型设计。我知道，生成器方法是如何工作的？
使用生成器，您可以返回一个响应，该响应基本上会在脚本或类似内容到达时缓慢地返回行（因此请求基本上需要很长时间，但几乎立即开始输出信息）。然后，您可以在客户端使用xhr调用该url（一次）来启动流程，并在结果发生时读取结果
这样做的风险是您无法重新启动它。如果客户端失去连接，他们将无法重新轮询端点，因为这将启动一个新进程，而不是与旧进程对话
知道了。是的，我在使用返回码之前做了类似的事情。
有一些策略可以绕过这一点，但很少值得付出努力。
好的，我将研究分叉和进程池。
非常感谢。
没问题
同样值得一提的是，对于几乎所有与金字塔有关的问题，这里可能都是合适的地方，这里的人知道所有的事情，他们通常都在附近。对于某些频道来说，情况并非总是如此，因此我认为值得一提。

这里有不止一种方法。此外，它不是Pyramid/Python独有的。一种方法（可能是最简单的方法，但不是最干净的方法）是将任务的进度保存到某个地方（如数据库），并在前端使用ajax请求进行轮询。一种更复杂的方法是使用消息队列、工作程序，以及一些将通知推送到客户端的方法（例如服务器发送的事件或WebSocket）。