Multithreading python中记录多线程死锁

我运行了10个进程，每个进程有10个线程，它们经常使用logging.info和logging.debug在30秒内不断地写入10个日志文件

有一次，通常是10秒后，会发生死锁。进程停止处理所有这些进程

带有py-bt&info线程的gdp-python[pid]显示它卡在这里：

  Id   Target Id                                 Frame 
* 1    Thread 0x7ff50f020740 (LWP 1622) "python" 0x00007ff50e8276d6 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x564f17c8aa80)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:205
  2    Thread 0x7ff509636700 (LWP 1624) "python" 0x00007ff50eb57bb7 in epoll_wait (epfd=8, events=0x7ff5096351d0, maxevents=256, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
  3    Thread 0x7ff508e35700 (LWP 1625) "python" 0x00007ff50eb57bb7 in epoll_wait (epfd=12, events=0x7ff508e341d0, maxevents=256, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
  4    Thread 0x7ff503fff700 (LWP 1667) "python" 0x00007ff50e8276d6 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x564f17c8aa80)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:205
...[threads 5-6 like 4]...
  7    Thread 0x7ff5027fc700 (LWP 1690) "python" 0x00007ff50eb46187 in __GI___libc_write (fd=2, buf=0x7ff50967bc24, nbytes=85) at ../sysdeps/unix/sysv/linux/write.c:27
...[threads 8-13 like 4]...

线程7的堆栈：

Traceback (most recent call first):
  File "/usr/lib/python2.7/logging/__init__.py", line 889, in emit
    stream.write(fs % msg)
...[skipped useless lines]...

这段代码我猜是记录函数的代码：

 884                                #the codecs module, but fail when writing to a
 885                                #terminal even when the codepage is set to cp1251.
 886                                #An extra encoding step seems to be needed.
 887                                stream.write((ufs % msg).encode(stream.encoding))
 888                        else:
>889                            stream.write(fs % msg)
 890                    except UnicodeError:
 891                        stream.write(fs % msg.encode("UTF-8"))
 892                self.flush()
 893            except (KeyboardInterrupt, SystemExit):
 894                raise

其余线程的堆栈类似-等待GIL：

Traceback (most recent call first):
  Waiting for the GIL
  File "/usr/lib/python2.7/threading.py", line 174, in acquire
    rc = self.__block.acquire(blocking)
  File "/usr/lib/python2.7/logging/__init__.py", line 715, in acquire
    self.lock.acquire()
...[skipped useless lines]...

据报道，包日志记录是多线程的，没有额外的锁。那么为什么包日志记录可能会死锁呢？它是否打开了太多的文件描述符或限制了其他内容

这就是我初始化它的方式，如果它很重要：

def get_logger(log_level, file_name='', log_name=''):
    if len(log_name) != 0:
        logger = logging.getLogger(log_name)
    else:
        logger = logging.getLogger()
    logger.setLevel(logger_state[log_level])
    formatter = logging.Formatter('%(asctime)s [%(levelname)s][%(name)s:%(funcName)s():%(lineno)s] - %(message)s')

    # file handler
    if len(file_name) != 0:
        fh = logging.FileHandler(file_name)
        fh.setLevel(logging.DEBUG)
        fh.setFormatter(formatter)
        logger.addHandler(fh)

    # console handler
    console_out = logging.StreamHandler()
    console_out.setLevel(logging.DEBUG)
    console_out.setFormatter(formatter)
    logger.addHandler(console_out)
    return logger

---

问题是因为我一直在将输出写入控制台和文件，但所有这些进程都是通过重定向到管道初始化的，而管道从未被监听过

            p = Popen(proc_params,
                      stdout=PIPE,
                      stderr=STDOUT,
                      close_fds=ON_POSIX,
                      bufsize=1
                      )

因此，在这种情况下，管道似乎有其缓冲区大小限制，并且在填充死锁之后

以下是解释：

这是为我不使用的函数而做的，但它似乎对Popen run有效，如果您不读取管道的话

Note

Do not use stdout=PIPE or stderr=PIPE with this function as that can deadlock based on the child process output volume. Use Popen with the communicate() method when you need pipes.