Java 冬眠“;明显的僵局!!!为未分配的挂起任务创建紧急线程&引用;

Java 冬眠“;明显的僵局!!!为未分配的挂起任务创建紧急线程&引用;,java,hibernate,c3p0,Java,Hibernate,C3p0,我最近在日志中收到了以下消息,随后出现了JVM崩溃(由于OOME)。我不知道该如何理解这一点,我真的希望得到任何指导 2015-03-19 21:15:02,457 [Timer-0] WARN (ThreadPoolAsynchronousRunner.java [run]:608) - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6824f21c -- APPARENT DEADLOCK!!! Cre

我最近在日志中收到了以下消息,随后出现了JVM崩溃(由于OOME)。我不知道该如何理解这一点,我真的希望得到任何指导

2015-03-19 21:15:02,457 [Timer-0] WARN  (ThreadPoolAsynchronousRunner.java [run]:608) - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6824f21c -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks! 
2015-03-19 21:26:29,543 [Timer-0] WARN  (ThreadPoolAsynchronousRunner.java [run]:624) - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6824f21c -- APPARENT DEADLOCK!!! Complete Status: 
    Managed Threads: 3
    Active Threads: 3
    Active Tasks: 
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@15da1b6b (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2)
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@b35b08a (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1)
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@51cfdd17 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0)
    Pending Tasks: 
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@19397937
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@5c7d3838
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@7aea62dd
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@55622ff2
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@74004a8
Pool thread stack traces:
    Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0,5,main]
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:560)
    Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2,5,main]
        java.net.SocketOutputStream.socketWrite0(Native Method)
        java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
        java.net.SocketOutputStream.write(SocketOutputStream.java:153)
        java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
        com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3227)
        com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1917)
        com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2060)
        com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2536)
        com.mysql.jdbc.ConnectionImpl.configureClientCharacterSet(ConnectionImpl.java:1751)
        com.mysql.jdbc.ConnectionImpl.initializePropsFromServer(ConnectionImpl.java:3425)
        com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2196)
        com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:718)
        com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
        sun.reflect.GeneratedConstructorAccessor306.newInstance(Unknown Source)
        sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
        com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:302)
        com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:282)
        com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:135)
        com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:182)
        com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:171)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:137)
        com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1014)
        com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:32)
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask.run(BasicResourcePool.java:1810)
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
    Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1,5,main]
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:560)


2015-03-19 21:56:59,137 [Timer-0] WARN  (ThreadPoolAsynchronousRunner.java [run]:608) - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6824f21c -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks! 
2015-03-19 21:56:59,143 [eXistThread-18676] ERROR (XQueryServlet.java [process]:566) - Java heap space 
java.lang.OutOfMemoryError: Java heap space
    at org.exist.storage.btree.BTree$BTreeNode.read(BTree.java:1269)
    at org.exist.storage.btree.BTree$BTreeNode.access$16(BTree.java:1239)
    at org.exist.storage.btree.BTree.getBTreeNode(BTree.java:460)
    at org.exist.storage.btree.BTree.scanSequential(BTree.java:413)
    at org.exist.storage.btree.BTree$BTreeNode.scanNextPage(BTree.java:2039)
    at org.exist.storage.btree.BTree$BTreeNode.query(BTree.java:1835)
    at org.exist.storage.btree.BTree$BTreeNode.query(BTree.java:1759)
    at org.exist.storage.btree.BTree$BTreeNode.query(BTree.java:1759)
    at org.exist.storage.btree.BTree$BTreeNode.query(BTree.java:1759)
    at org.exist.storage.btree.BTree$BTreeNode.access$12(BTree.java:1734)
    at org.exist.storage.btree.BTree.query(BTree.java:379)
    at org.exist.storage.structural.NativeStructuralIndexWorker.scanByType(NativeStructuralIndexWorker.java:259)
    at org.exist.dom.VirtualNodeSet.getNodesFromIndex(VirtualNodeSet.java:457)
    at org.exist.dom.VirtualNodeSet.realize(VirtualNodeSet.java:585)
    at org.exist.dom.VirtualNodeSet.iterator(VirtualNodeSet.java:860)
    at org.exist.dom.AbstractNodeSet.iterator(AbstractNodeSet.java:1)
    at org.exist.storage.structural.NativeStructuralIndexWorker.findDescendantsByTagName(NativeStructuralIndexWorker.java:162)
    at org.exist.xquery.LocationStep.getAttributes(LocationStep.java:645)
    at org.exist.xquery.LocationStep.eval(LocationStep.java:434)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.Predicate.selectByNodeSet(Predicate.java:446)
    at org.exist.xquery.Predicate.evalPredicate(Predicate.java:326)
    at org.exist.xquery.LocationStep.processPredicate(LocationStep.java:251)
    at org.exist.xquery.LocationStep.applyPredicate(LocationStep.java:238)
    at org.exist.xquery.LocationStep.eval(LocationStep.java:462)
    at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
    at org.exist.xquery.PathExpr.eval(PathExpr.java:264)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:142)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.LetExpr.eval(LetExpr.java:187)
    at org.exist.xquery.BindingExpression.eval(BindingExpression.java:164)
2015-03-19 21:56:59,147 [Timer-0] WARN  (ThreadPoolAsynchronousRunner.java [run]:624) - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6824f21c -- APPARENT DEADLOCK!!! Complete Status: 
    Managed Threads: 3
    Active Threads: 3
    Active Tasks: 
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@79180a12 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0)
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@243c6d0c (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1)
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@50191373 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2)
    Pending Tasks: 
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@3a9d08ca
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@3ecdd11
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@44ff846d
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@5ce5850a
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@eec1d04
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@6b8d4d9d
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@53e9706d
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@23d472cf
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@4dbe4f8c
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@4c5e0203
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@54ac79fd
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@546e2bad
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@6b13cc83
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@57e185f8
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@60357d68
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@45231180
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@3021aa73
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@6bb437ca
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@2021c9e9
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@7d53637c
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@409c2c97
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@adc5929
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@241ca71a
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@42b26866
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@636b1c33
        com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask@b160466
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@4af34669
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@1b53e609
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@2062ebd4
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@1b6cfe8a
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@4b7c2380
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@4f9be748
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@78108924
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@474b002
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@2ebee32f
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@3e0fe017
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@42aa175b
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@637f5bac
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@3a017b77
        com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask@7b4f2b78
Pool thread stack traces:
    Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1,5,main]
        sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
        com.mysql.jdbc.ResultSetImpl.getInstance(ResultSetImpl.java:370)
        com.mysql.jdbc.MysqlIO.buildResultSetWithRows(MysqlIO.java:2532)
        com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:477)
        com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:2510)
        com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1746)
        com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2135)
        com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2536)
        com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2465)
        com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1383)
        com.mysql.jdbc.DatabaseMetaData$9.forEach(DatabaseMetaData.java:4826)
        com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:50)
        com.mysql.jdbc.DatabaseMetaData.getTables(DatabaseMetaData.java:4804)
        com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnectionNoQuery(DefaultConnectionTester.java:185)
        com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:62)
        com.mchange.v2.c3p0.AbstractConnectionTester.activeCheckConnection(AbstractConnectionTester.java:67)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:368)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.java:310)
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.java:1999)
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
    Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2,5,main]
        com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2596)
        com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2465)
        com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1383)
        com.mysql.jdbc.DatabaseMetaData$9.forEach(DatabaseMetaData.java:4826)
        com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:50)
        com.mysql.jdbc.DatabaseMetaData.getTables(DatabaseMetaData.java:4804)
        com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnectionNoQuery(DefaultConnectionTester.java:185)
        com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:62)
        com.mchange.v2.c3p0.AbstractConnectionTester.activeCheckConnection(AbstractConnectionTester.java:67)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:368)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.java:310)
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.java:1999)
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
    Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0,5,main]
        java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1012)
        java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
        com.newrelic.agent.TransactionService.addTransaction(TransactionService.java:142)
        com.newrelic.agent.Transaction.getTransaction(Transaction.java:1104)
        com.newrelic.agent.Transaction.getTransaction(Transaction.java:1087)
        com.newrelic.agent.TracerService$TracerServiceImpl.getTracer(TracerService.java:136)
        com.newrelic.agent.TracerService.getTracer(TracerService.java:41)
        com.newrelic.agent.instrumentation.InvocationPoint.invoke(InvocationPoint.java:55)
        com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java)
        com.mysql.jdbc.DatabaseMetaData$9.forEach(DatabaseMetaData.java:4826)
        com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:50)
        com.mysql.jdbc.DatabaseMetaData.getTables(DatabaseMetaData.java:4804)
        com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnectionNoQuery(DefaultConnectionTester.java:185)
        com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:62)
        com.mchange.v2.c3p0.AbstractConnectionTester.activeCheckConnection(AbstractConnectionTester.java:67)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:368)
        com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.java:310)
        com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.java:1999)
        com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)

服务器的负载非常轻,每秒有5次查询。(Java 8)

因此,直接的问题是连接池试图获取新的连接,但是试图获取这些任务的任务冻结了很长一段时间,以至于c3p0决定任务必须死锁,然后丢弃并替换线程池。后来,线程池挂起在空闲连接测试任务上

通常,“挂起”任务看起来像第一个“池线程堆栈跟踪:”标签下面的第二个线程:执行无法完成的网络IO。您的情况很奇怪,三个线程中有两个没有卡在IO中。他们刚刚开始做任何事情,但他们还没有活下来。然后您会遇到OutOfMemoryError,并且在空闲连接测试任务上会出现另一个明显的死锁,这些任务看起来也应该是活动的

也许您的应用程序非常接近某种资源限制,导致运行非常缓慢?直截了当地说,您可以增加此应用程序的可用内存量(或者修改它以降低内存占用)。当线程池第二次尝试刷新和重新创建挂起的线程时,您会遇到OOME,这并不是直接引起的,但很可能是由不断增长的线程占用引起的。(在您的日志中,以前是否有很多明显的死锁?如果强制执行JVM线程转储,您是否会看到大量的
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread
实例仍然挂在旧的、未完成的任务上

一些一般性意见:如果您的负载不超过5个同步查询,为什么池会这么重?为什么不
min_size
5-ish,max 10ish?对于池中允许的连接数,您的max_语句设置太小了。我会完全忽略这一点,直到您让事情顺利进行。然后,为了更好地执行因此,如果需要,您可以设置关于
maxStatementsPerConnection
参数的更直接的推理


大多数情况下,您需要保留应用程序的占用空间(内存?线程?)远远低于分配给它的资源,或者通过增加资源、减少其占用空间,或者修复可能存在的导致其资源占用空间增加到极限的任何问题。我将首先使池变小,可用内存变大,并将池配置得更小。

我遇到了相同的异常并原因是db用户的密码错误


c3p0对我来说,你也是一个有趣的家伙。

身份验证问题。我刚刚将域部分添加到数据库服务器名称中,一切正常。这是一个非常误导性的错误。

非常感谢你的详细回答。为了回答你的问题,这是我第一次在日志中看到这些消息。我想你可能是对的,这就是李kely是OOME异常的症状,而不仅仅是c3p0问题。我将设置一些详细的监视,看看它告诉我有关资源使用情况的信息。谢谢!“我将首先使池更小,可用内存更大,并将池配置得更小。”使池变小和将池配置得更小有什么区别。是的,没错,我也遇到了这个问题。当数据库关闭时,我收到了这个错误消息,我希望我的情况就是这么简单。我希望你没有在得到答案后删除列表中的问题,因为这不好;)
 <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-core</artifactId>
            <version>4.3.6.Final</version>
        </dependency>

        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-c3p0</artifactId>
            <version>4.3.6.Final</version>
        </dependency>
  configuration = new Configuration().setProperty("hibernate.dialect", "org.hibernate.dialect.MySQL5InnoDBDialect")
  .setProperty("hibernate.connection.provider_class", "org.hibernate.connection.C3P0ConnectionProvider")
  .setProperty("hibernate.c3p0.idle_test_period", "1000")
  .setProperty("hibernate.c3p0.min_size", "20")
  .setProperty("hibernate.c3p0.max_size", "50")
  .setProperty("hibernate.c3p0.timeout", "1800")
  .setProperty("hibernate.c3p0.max_statements", "50")