Kotlin 如何正确配置HikariCP以处理必须保持活动状态的连接?

Kotlin 如何正确配置HikariCP以处理必须保持活动状态的连接?,kotlin,spring-data-jpa,spring-data,hikaricp,pgjdbc-ng,Kotlin,Spring Data Jpa,Spring Data,Hikaricp,Pgjdbc Ng,我使用Spring Boot 2.4.0和Spring Boot数据JPA连接到PostgreSQL,并使用基于JPA的存储库执行典型的读写操作。因为数据库也被其他服务使用,所以我使用LISTEN/NOTIFY功能()来通知来自PostgeSQL的更改。为此,我使用驱动程序com.impossibl.postgres.jdbc.PGDriver而不是默认驱动程序和以下代码使Spring侦听数据库的更改: @Service class PostgresChangeListener( val

我使用Spring Boot 2.4.0和Spring Boot数据JPA连接到PostgreSQL,并使用基于JPA的存储库执行典型的读写操作。因为数据库也被其他服务使用,所以我使用LISTEN/NOTIFY功能()来通知来自PostgeSQL的更改。为此,我使用驱动程序
com.impossibl.postgres.jdbc.PGDriver
而不是默认驱动程序和以下代码使Spring侦听数据库的更改:

@Service
class PostgresChangeListener(
    val dataSource: HikariDataSource,
    @Qualifier("dbToPGReceiverQueue") val postgresQueue: RBlockingQueue<String>
) {
    init {
        listenToNotifyMessage()
    }

    final fun listenToNotifyMessage() {
        val notificationListener = object:PGNotificationListener {
            override fun notification(processId: Int, channelName: String, payload: String) {
                log.info("Received change from PostgresQL: $processId, $channelName, $payload")
                postgresQueue.add(payload)
            }
            override fun closed() {
                log.debug("Connection to Postgres lost! Try to reconnect...")
                listenToNotifyMessage()
            }
        }

        try {
            val connection = DataSourceUtils.getConnection(dataSource).unwrap(PGConnection::class.java)
            connection.addNotificationListener(notificationListener)
            connection.createStatement().use { statement -> statement.execute("LISTEN change_notifier;") }
        } catch (e: SQLException) {
            throw RuntimeException(e)
        }
    }
}

为了找到问题,我按照上的建议启用了Hikari日志记录。以下是日志摘录的输出:

2021-03-02 21:31:59.055 DEBUG 1 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=10, active=1, idle=9, waiting=0)
...
2021-03-02 21:31:59.055 DEBUG 1 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=10, active=1, idle=9, waiting=0)
2021-03-02 22:00:53.139 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.impossibl.postgres.jdbc.PGDirectConnection@201ab69f: (connection has passed maxLifetime)
2021-03-02 22:00:53.162 DEBUG 1 --- [onnection adder] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection com.impossibl.postgres.jdbc.PGDirectConnection@f2ffd1ea
2021-03-02 22:00:54.709 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.impossibl.postgres.jdbc.PGDirectConnection@3bb847ef: (connection has passed maxLifetime)
2021-03-02 22:00:54.730 DEBUG 1 --- [onnection adder] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection com.impossibl.postgres.jdbc.PGDirectConnection@fd5932d7
2021-03-02 22:00:59.110 DEBUG 1 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=10, active=1, idle=9, waiting=0)
2021-03-02 22:00:59.111 DEBUG 1 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Fill pool skipped, pool is at sufficient level.
2021-03-02 22:01:04.782 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.impossibl.postgres.jdbc.PGDirectConnection@1d081266: (connection has passed maxLifetime)
2021-03-02 22:01:04.803 DEBUG 1 --- [onnection adder] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection com.impossibl.postgres.jdbc.PGDirectConnection@e0b396bc
2021-03-02 22:01:09.295 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.impossibl.postgres.jdbc.PGDirectConnection@a2b0bd29: (connection has passed maxLifetime)
2021-03-02 22:01:09.313 DEBUG 1 --- [onnection adder] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection com.impossibl.postgres.jdbc.PGDirectConnection@ca9c8226
2021-03-02 22:01:10.075 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.impossibl.postgres.jdbc.PGDirectConnection@ec8746aa: (connection has passed maxLifetime)
2021-03-02 22:01:10.093 DEBUG 1 --- [onnection adder] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection com.impossibl.postgres.jdbc.PGDirectConnection@aff2bfd8
2021-03-02 22:01:12.820 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.impossibl.postgres.jdbc.PGDirectConnection@a7e0fc39: (connection has passed maxLifetime)
2021-03-02 22:01:12.840 DEBUG 1 --- [onnection adder] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection com.impossibl.postgres.jdbc.PGDirectConnection@d637554
2021-03-02 22:01:15.099 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.impossibl.postgres.jdbc.PGDirectConnection@dadcba66: (connection has passed maxLifetime)
2021-03-02 22:01:15.119 DEBUG 1 --- [onnection adder] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection com.impossibl.postgres.jdbc.PGDirectConnection@e29805ef
2021-03-02 22:01:21.558 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.impossibl.postgres.jdbc.PGDirectConnection@762f0753: (connection has passed maxLifetime)
2021-03-02 22:01:21.576 DEBUG 1 --- [onnection adder] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection com.impossibl.postgres.jdbc.PGDirectConnection@d5b8d008
2021-03-02 22:01:23.351 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.impossibl.postgres.jdbc.PGDirectConnection@5e4721b0: (connection has passed maxLifetime)
2021-03-02 22:01:23.370 DEBUG 1 --- [onnection adder] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection com.impossibl.postgres.jdbc.PGDirectConnection@a8606b56
2021-03-02 22:01:29.111 DEBUG 1 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=10, active=1, idle=9, waiting=0)
2021-03-02 22:01:29.111 DEBUG 1 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Fill pool skipped, pool is at sufficient level.
2021-03-02 22:01:59.112 DEBUG 1 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=10, active=1, idle=9, waiting=0)
...
对我来说,日志看起来是正确的,但过了一段时间,活动连接越来越多

...
2021-03-03 06:31:29.664 DEBUG 1 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=10, active=9, idle=1, waiting=0)
2021-03-03 06:31:48.687 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.impossibl.postgres.jdbc.PGDirectConnection@4fa5ec41: (connection is dead)
2021-03-03 06:31:48.707 DEBUG 1 --- [onnection adder] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection com.impossibl.postgres.jdbc.PGDirectConnection@693052fe
2021-03-03 06:31:48.709 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Fill pool skipped, pool is at sufficient level.
2021-03-03 06:31:59.665 DEBUG 1 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Pool stats (total=10, active=10, idle=0, waiting=1)
2021-03-03 06:31:59.665 DEBUG 1 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Fill pool skipped, pool is at sufficient level.
2021-03-03 06:32:20.199 DEBUG 1 --- [io-8080-exec-10] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Timeout failure stats (total=10, active=10, idle=0, waiting=2)
2021-03-03 06:32:20.208  WARN 1 --- [io-8080-exec-10] o.s.b.a.jdbc.DataSourceHealthIndicator   : DataSource health check failed

org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms.
...
。。。直到出现所描述的错误消息


我想知道如何正确配置Hikari或更改代码以避免所述错误?我希望你能帮忙。

不是同一个问题,但我有一个类似的问题。当我的数据库重新启动时,Hikari无法关闭活动的侦听器连接,整个通知停止工作

我找到了一个可能的解决办法。Hikari无法在连接断开时关闭连接的原因是,您正在从此处的代理连接中展开连接:

DataSourceUtils.getConnection(dataSource).unwrap(PGConnection::class.java)
在此之后,将notificationListener附加到PGConnection,使其保持活动状态

首先要避免hikaripool泄漏,您应该分离2连接,初始化侦听器后,您应该关闭hikaripool连接

private hikariConnection: Connection;

...

hikariConnection = DataSourceUtils.getConnection(dataSource)
val pgConnection: PGConnection = hikariConnection.unwrap(PGConnection::class.java)

... init the listener
hikariConnection.close()

在PGNotificationListener.closed()中,您必须将侦听器重新斜体化,从数据源获取新连接。但是要小心,在Hikaripool填充其池时获得新连接(因为数据库中断仅几秒钟),可能会相互阻塞。我们通过在一个专用的新线程上建立新连接来解决这个问题

 override fun closed() {
            ... get a new PGConnection, and start listening for the notifications
    }

抱歉,如果它没有正确回答您的问题,但它可能会对某些人有所帮助。

我想您已经准确地回答了我的问题!事实上,我认为连接必须始终处于活动状态,因为我必须侦听数据库。但与我的预期相反,侦听器不需要任何活动连接。连接数量不断增加可能是因为数据库暂时不可用。由于您刚才描述的原因,可能没有返回连接。我还有一个问题,即无法通过关闭的函数直接建立新连接。您确实必须通过线程或kotlin例程退出函数。非常感谢您的准确回答!不客气!这是一个棘手的问题。
 override fun closed() {
            ... get a new PGConnection, and start listening for the notifications
    }