Database PostgreSQL-连接到新数据库的速度非常慢
我刚刚安装了Postgres(v12.2)并创建了一个数据库,但是连接(使用psql)大约需要20秒。这似乎是一个很长的时间,它造成了我的一些工作流程的问题 我使用Anaconda安装了PostgreSQL(出于公司原因,我无法访问apt或Docker)。以下是我启动并运行数据库所遵循的步骤:Database PostgreSQL-连接到新数据库的速度非常慢,database,postgresql,performance,database-connection,Database,Postgresql,Performance,Database Connection,我刚刚安装了Postgres(v12.2)并创建了一个数据库,但是连接(使用psql)大约需要20秒。这似乎是一个很长的时间,它造成了我的一些工作流程的问题 我使用Anaconda安装了PostgreSQL(出于公司原因,我无法访问apt或Docker)。以下是我启动并运行数据库所遵循的步骤: conda install postgresql initdb -D airflow pg_ctl -D airflow -l logfile start createuser --encrypted -
conda install postgresql
initdb -D airflow
pg_ctl -D airflow -l logfile start
createuser --encrypted --pwprompt airflow_user
createdb --owner=airflow_user airflow_db
我还更改了pg_hba.conf文件中的以下行:
# IPv4 local connections:
host all all 0.0.0.0/0 trust
并在postgresql.conf文件中将日志设置为尽可能详细(还向日志消息添加了应用程序名称等内容)
使用以下各项测试连接:
psql -h 127.0.0.1 -p 5432 -d airflow_db -U airflow_user
生成以下日志消息:
2021-02-17 15:56:25.590 UTC [unknown] [unknown] LOG: connection received: host=127.0.0.1 port=45006
2021-02-17 15:56:41.454 UTC DEBUG: StartTransaction(1) name: unnamed; blockState: DEFAULT; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:41.454 UTC DEBUG: CommitTransaction(1) name: unnamed; blockState: STARTED; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:41.455 UTC DEBUG: received inquiry for database 0
2021-02-17 15:56:41.455 UTC DEBUG: writing stats file "pg_stat_tmp/global.stat"
2021-02-17 15:56:41.455 UTC DEBUG: writing stats file "pg_stat_tmp/db_0.stat"
2021-02-17 15:56:41.466 UTC DEBUG: InitPostgres
2021-02-17 15:56:41.466 UTC DEBUG: my backend ID is 3
2021-02-17 15:56:41.466 UTC DEBUG: StartTransaction(1) name: unnamed; blockState: DEFAULT; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:41.467 UTC DEBUG: CommitTransaction(1) name: unnamed; blockState: STARTED; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:41.467 UTC DEBUG: autovacuum: processing database "airflow_db"
2021-02-17 15:56:41.467 UTC DEBUG: received inquiry for database 16385
2021-02-17 15:56:41.467 UTC DEBUG: writing stats file "pg_stat_tmp/global.stat"
2021-02-17 15:56:41.467 UTC DEBUG: writing stats file "pg_stat_tmp/db_16385.stat"
2021-02-17 15:56:41.467 UTC DEBUG: writing stats file "pg_stat_tmp/db_0.stat"
2021-02-17 15:56:41.477 UTC DEBUG: StartTransaction(1) name: unnamed; blockState: DEFAULT; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:41.477 UTC DEBUG: pg_statistic: vac: 0 (threshold 134), anl: 0 (threshold 92)
2021-02-17 15:56:41.478 UTC DEBUG: pg_type: vac: 0 (threshold 131), anl: 0 (threshold 91)
2021-02-17 15:56:41.478 UTC DEBUG: pg_authid: vac: 0 (threshold 52), anl: 1 (threshold 51)
2021-02-17 15:56:41.478 UTC DEBUG: pg_attribute: vac: 0 (threshold 633), anl: 0 (threshold 341)
2021-02-17 15:56:41.478 UTC DEBUG: pg_proc: vac: 0 (threshold 642), anl: 0 (threshold 346)
2021-02-17 15:56:41.478 UTC DEBUG: pg_class: vac: 0 (threshold 129), anl: 0 (threshold 90)
2021-02-17 15:56:41.478 UTC DEBUG: pg_index: vac: 0 (threshold 82), anl: 0 (threshold 66)
2021-02-17 15:56:41.478 UTC DEBUG: pg_operator: vac: 0 (threshold 204), anl: 0 (threshold 127)
2021-02-17 15:56:41.478 UTC DEBUG: pg_opclass: vac: 0 (threshold 76), anl: 0 (threshold 63)
2021-02-17 15:56:41.478 UTC DEBUG: pg_am: vac: 0 (threshold 51), anl: 0 (threshold 51)
2021-02-17 15:56:41.478 UTC DEBUG: pg_amop: vac: 0 (threshold 193), anl: 0 (threshold 122)
2021-02-17 15:56:41.478 UTC DEBUG: pg_amproc: vac: 0 (threshold 139), anl: 0 (threshold 95)
2021-02-17 15:56:41.478 UTC DEBUG: pg_rewrite: vac: 0 (threshold 75), anl: 0 (threshold 63)
2021-02-17 15:56:41.478 UTC DEBUG: pg_cast: vac: 0 (threshold 93), anl: 0 (threshold 72)
2021-02-17 15:56:41.478 UTC DEBUG: pg_namespace: vac: 0 (threshold 51), anl: 0 (threshold 51)
2021-02-17 15:56:41.478 UTC DEBUG: pg_database: vac: 0 (threshold 50), anl: 1 (threshold 50)
2021-02-17 15:56:41.478 UTC DEBUG: pg_tablespace: vac: 0 (threshold 50), anl: 0 (threshold 50)
2021-02-17 15:56:41.478 UTC DEBUG: pg_shdepend: vac: 0 (threshold 52), anl: 1 (threshold 51)
2021-02-17 15:56:41.478 UTC DEBUG: pg_toast_2618: vac: 0 (threshold 100), anl: 0 (threshold 75)
2021-02-17 15:56:41.478 UTC DEBUG: CommitTransaction(1) name: unnamed; blockState: STARTED; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:41.478 UTC DEBUG: shmem_exit(0): 1 before_shmem_exit callbacks to make
2021-02-17 15:56:41.478 UTC DEBUG: shmem_exit(0): 7 on_shmem_exit callbacks to make
2021-02-17 15:56:41.478 UTC DEBUG: proc_exit(0): 2 callbacks to make
2021-02-17 15:56:41.478 UTC DEBUG: exit(0)
2021-02-17 15:56:41.478 UTC DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:56:41.478 UTC DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:56:41.478 UTC DEBUG: proc_exit(-1): 0 callbacks to make
2021-02-17 15:56:41.478 UTC DEBUG: reaping dead processes
2021-02-17 15:56:41.478 UTC DEBUG: server process (PID 29567) exited with exit code 0
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG: shmem_exit(0): 0 before_shmem_exit callbacks to make
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG: shmem_exit(0): 0 on_shmem_exit callbacks to make
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG: proc_exit(0): 1 callbacks to make
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG: exit(0)
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG: proc_exit(-1): 0 callbacks to make
2021-02-17 15:56:47.647 UTC DEBUG: forked new backend, pid=29613 socket=9
2021-02-17 15:56:47.647 UTC DEBUG: reaping dead processes
2021-02-17 15:56:47.647 UTC DEBUG: server process (PID 29413) exited with exit code 0
2021-02-17 15:56:47.647 UTC [unknown] [unknown] LOG: connection received: host=127.0.0.1 port=45008
2021-02-17 15:56:47.647 UTC [unknown] airflow_user DEBUG: postgres child[29613]: starting with (
2021-02-17 15:56:47.647 UTC [unknown] airflow_user DEBUG: postgres
2021-02-17 15:56:47.647 UTC [unknown] airflow_user DEBUG: )
2021-02-17 15:56:47.647 UTC [unknown] airflow_user DEBUG: InitPostgres
2021-02-17 15:56:47.648 UTC [unknown] airflow_user DEBUG: my backend ID is 3
2021-02-17 15:56:47.648 UTC [unknown] airflow_user DEBUG: StartTransaction(1) name: unnamed; blockState: DEFAULT; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:47.648 UTC [unknown] airflow_user LOG: connection authorized: user=airflow_user database=airflow_db application_name=psql
2021-02-17 15:56:47.649 UTC psql airflow_user DEBUG: CommitTransaction(1) name: unnamed; blockState: STARTED; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG: shmem_exit(0): 1 before_shmem_exit callbacks to make
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG: shmem_exit(0): 6 on_shmem_exit callbacks to make
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG: proc_exit(0): 4 callbacks to make
2021-02-17 15:56:50.746 UTC psql airflow_user LOG: disconnection: session time: 0:00:03.098 user=airflow_user database=airflow_db host=127.0.0.1 port=45008
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG: exit(0)
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG: proc_exit(-1): 0 callbacks to make
2021-02-17 15:56:50.746 UTC DEBUG: reaping dead processes
2021-02-17 15:56:50.746 UTC DEBUG: server process (PID 29613) exited with exit code 0
2021-02-17 15:57:00.668 UTC DEBUG: postmaster received signal 2
2021-02-17 15:57:00.668 UTC LOG: received fast shutdown request
2021-02-17 15:57:00.668 UTC LOG: aborting any active transactions
2021-02-17 15:57:00.668 UTC DEBUG: sending signal 15 to process 26962
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(0): 1 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: autovacuum launcher shutting down
2021-02-17 15:57:00.668 UTC DEBUG: logical replication launcher shutting down
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(0): 5 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(1): 2 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(0): 1 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: proc_exit(0): 2 callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(0): 6 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: exit(0)
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(1): 6 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: proc_exit(0): 2 callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: proc_exit(1): 2 callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: exit(0)
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(0): 1 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: exit(1)
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(0): 5 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: proc_exit(0): 2 callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: exit(0)
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC DEBUG: proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.669 UTC DEBUG: reaping dead processes
2021-02-17 15:57:00.669 UTC LOG: background worker "logical replication launcher" (PID 26962) exited with exit code 1
2021-02-17 15:57:00.669 UTC DEBUG: reaping dead processes
2021-02-17 15:57:00.669 UTC LOG: shutting down
2021-02-17 15:57:00.670 UTC DEBUG: performing replication slot checkpoint
2021-02-17 15:57:00.672 UTC DEBUG: attempting to remove WAL segments older than log file 000000000000000000000000
2021-02-17 15:57:00.672 UTC DEBUG: SlruScanDirectory invoking callback on pg_subtrans/0000
2021-02-17 15:57:00.672 UTC DEBUG: shmem_exit(0): 1 before_shmem_exit callbacks to make
2021-02-17 15:57:00.672 UTC DEBUG: shmem_exit(0): 5 on_shmem_exit callbacks to make
2021-02-17 15:57:00.672 UTC DEBUG: proc_exit(0): 2 callbacks to make
2021-02-17 15:57:00.672 UTC DEBUG: exit(0)
2021-02-17 15:57:00.672 UTC DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.672 UTC DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.672 UTC DEBUG: proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.672 UTC DEBUG: reaping dead processes
2021-02-17 15:57:00.672 UTC DEBUG: writing stats file "pg_stat/global.stat"
2021-02-17 15:57:00.672 UTC DEBUG: writing stats file "pg_stat/db_16385.stat"
2021-02-17 15:57:00.673 UTC DEBUG: removing temporary stats file "pg_stat_tmp/db_16385.stat"
2021-02-17 15:57:00.673 UTC DEBUG: writing stats file "pg_stat/db_12738.stat"
2021-02-17 15:57:00.673 UTC DEBUG: removing temporary stats file "pg_stat_tmp/db_12738.stat"
2021-02-17 15:57:00.673 UTC DEBUG: writing stats file "pg_stat/db_0.stat"
2021-02-17 15:57:00.673 UTC DEBUG: removing temporary stats file "pg_stat_tmp/db_0.stat"
2021-02-17 15:57:00.673 UTC DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.673 UTC DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.673 UTC DEBUG: proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.673 UTC DEBUG: reaping dead processes
2021-02-17 15:57:00.673 UTC DEBUG: shmem_exit(0): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.673 UTC DEBUG: shmem_exit(0): 5 on_shmem_exit callbacks to make
2021-02-17 15:57:00.673 UTC DEBUG: cleaning up dynamic shared memory control segment with ID 521889379
2021-02-17 15:57:00.676 UTC DEBUG: proc_exit(0): 2 callbacks to make
2021-02-17 15:57:00.676 UTC LOG: database system is shut down
2021-02-17 15:57:00.676 UTC DEBUG: exit(0)
2021-02-17 15:57:00.676 UTC DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.676 UTC DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.676 UTC DEBUG: proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.677 UTC DEBUG: logger shutting down
2021-02-17 15:57:00.677 UTC DEBUG: shmem_exit(0): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.677 UTC DEBUG: shmem_exit(0): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.677 UTC DEBUG: proc_exit(0): 0 callbacks to make
2021-02-17 15:57:00.677 UTC DEBUG: exit(0)
2021-02-17 15:57:00.677 UTC DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.677 UTC DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.677 UTC DEBUG: proc_exit(-1): 0 callbacks to make
我注意到的一件事是,从“connection received”(连接已接收)消息到下一步,大约需要11秒。我已经在我的个人桌面(不工作)上测试过了,连接几乎是即时的,这似乎表明这是某种环境问题
如果有人有任何见解,我很想听听!我已经被封锁了很久了
非常感谢 问题解决了。Anaconda的PostgreSQL包有问题,导致每次连接尝试失败一次,然后在第二次尝试中成功。不知道到底发生了什么,但我们使用良好的旧版本从源代码构建了PostgreSQL,它工作起来很有魅力
为大家干杯日志中混合了太多内容,并且没有进程号(
%p
)。