Python 如何检测django应用程序中的死锁（并消除它们）

我正在维护一个django项目，该项目定期没有响应。到目前为止，我通过持续监视应用程序并在必要时重新启动apache来处理这种情况

反应迟钝怎么办？这意味着apache不再回复任何请求

环境：

• OS:Debian压缩64位
• Webserver:apache2.2.16 mod_wsgi（mod_python投入生产大约一年）
• Django:1.3.1（以及自1.0以来的每个主要版本）
• Python:2.6.6+virtualenv（使用distribute，没有站点包，以前在生产中使用过几种不同的设置）
• 数据库后端：psycopg2.3.2
• 数据库：PostgreSQL 9.0（过去使用的是8.3版）
• 连接池：pgbouncer（如果不使用bouncer，问题仍然存在）
• 反向代理：nginx 1.0.11

如何才能更接近错误的根源？（我不能提供源代码-这里有代码片段，但也有可能） 我一直在寻找这个问题，所以不可能列出所有我尝试过的东西。我试图摆脱我能想到的任何“魔法”。问题发生后，应用程序的几个部分已被重写

---

很抱歉没有提供详细信息，但我很乐意（几乎）提供所需的任何信息，并承诺尽我所能使此帖子尽可能对面临类似问题的其他人有所帮助。

最终，您需要添加到mod_wsgi 4.0中的新功能。这将允许守护进程模式在请求阻塞时更好地控制自动重启。在阻塞条件下重新启动时，mod_wsgi将尝试转储每个Python请求线程当时正在执行的Python堆栈跟踪，以便您可以看到它们被阻塞的原因

建议您在mod_wsgi邮件列表上讨论这个问题，如果需要，可以更详细地解释新特性。以前曾在以下网站发布过相关信息：

---

mod_wsgi 4.0代码目前只能从源代码存储库中获得。当前的树干头被认为是稳定的。

您可能被以下django bug[1]咬伤（1.4 branch中尚未修复）

解决方法：手动将应用于django源，或在wsgi模块周围使用线程安全包装，如下所示（我们在生产系统中使用此包装）

---

[1]

您在进行什么样的监控？Munin、Monit、Nagios？相关的监视通过一个shell脚本完成，该脚本每30秒检查一次服务器状态和静态页面。我还有用于操作统计（请求数等）的munin和用于监视其他一些所需资源的nagios。settings.py中确实有

DEBUG=False

，对吗？如果使用Nginx作为反向代理，为什么要使用完整的Apache堆栈？@danodonovan当然

DEBUG=False

from __future__ import with_statement
from  django.core.handlers.wsgi import WSGIHandler as DjangoWSGIHandler

from threading import Lock

__copyright__ = "Jibe"

class WSGIHandler(DjangoWSGIHandler):
    """
    This provides a threadsafe drop-in replacement of django's WSGIHandler.

    Initialisation of django via a multithreaded wsgi handler is not safe.
    It is vulnerable to a A-B B-A deadlock.

When two threads bootstrap django via different urls you have a change to hit 
the following deadlock.

  thread 1                                               thread  2
    view A                                                  view B
     import file foo            import lock foo               import file bar  import lock bar
           bootstrap django     lock AppCache.write_lock
                import file bar import lock bar  <-- blocks
                                                                 bootstrap django    lock AppCache.write_lock  <----- deadlock

workaround for an AB BA deadlock:  wrap it in a lock C.

        lock C                      lock C
            lock A                      lock B
            lock B                      lock A
            release B                   release A
            release A                   release A
        release C                   release C          

    Thats exactly what this class does,  but... only for the first few calls.  
    After that we remove the lock C.  as the AppCache.write_lock is only held when django is booted. 

    If we would not remove the lock C after the first few calls, that would make the whole app single threaded again. 

    Usage:    
        in your wsgi file replace   the following lines 
                import django.core.handlers.wsgi.WSGIHandler  
                application = django.core.handlers.wsgi.WSGIHandler 
        by 
                import threadsafe_wsgi 
                application = threadsafe_wsgi.WSGIHandler 


    FAQ: 
        Q: why would you want threading in the first place ?                 
        A: to reduce memory. Big apps can consume hundeds of megabytes each.  adding processes is then much more expensive than threads. 
           that memory is better spend caching, when threads are almost free. 

        Q: this deadlock, it looks far-fetched, is this real ? 
        A: yes we had this problem on production machines. 
    """ 
    __initLock = Lock()  # lock C 
    __initialized = 0 

    def __call__(self, environ, start_response): 
        # the first calls (4) we squeeze everybody through lock C 
        # this basically serializes all threads 
        MIN_INIT_CALLS = 4 
        if self.__initialized < MIN_INIT_CALLS: 
            with self.__initLock: 
                ret = DjangoWSGIHandler.__call__(self, environ, start_response) 
                self.__initialized += 1 
                return ret 
        else: 
            # we are safely bootrapped, skip lock C 
            # now we are running multi-threaded again 
            return  DjangoWSGIHandler.__call__(self, environ, start_response)

from threadsafe_wsgi.handlers import WSGIHandler
django_handler = WSGIHandler()