PostgreSQL - 连接到新数据库的速度很慢

时间:2021-02-17 14:33:17

标签: database postgresql performance database-connection

我刚刚安装了 Postgres (v12.2) 并创建了一个数据库,但连接(使用 psql)大约需要 20 秒。这似乎是一个很长的时间,它在我的一些工作流程中造成了问题。

我使用 Anaconda 安装了 PostgreSQL(出于公司原因,我无法访问 apt 或 Docker)。以下是我启动和运行数据库所遵循的步骤:

conda install postgresql
initdb -D airflow
pg_ctl -D airflow -l logfile start
createuser --encrypted --pwprompt airflow_user
createdb --owner=airflow_user airflow_db

我还在 pg_hba.conf 文件中更改了以下几行:

# IPv4 local connections:
host    all             all             0.0.0.0/0            trust

并在 postgresql.conf 文件中将日志设置为尽可能详细(还在日志消息中添加了一些诸如应用程序名称之类的内容)。

测试连接:

psql -h 127.0.0.1 -p 5432 -d airflow_db -U airflow_user

产生以下日志消息:

2021-02-17 15:56:25.590 UTC [unknown] [unknown] LOG:  connection received: host=127.0.0.1 port=45006
2021-02-17 15:56:41.454 UTC   DEBUG:  StartTransaction(1) name: unnamed; blockState: DEFAULT; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:41.454 UTC   DEBUG:  CommitTransaction(1) name: unnamed; blockState: STARTED; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:41.455 UTC   DEBUG:  received inquiry for database 0
2021-02-17 15:56:41.455 UTC   DEBUG:  writing stats file "pg_stat_tmp/global.stat"
2021-02-17 15:56:41.455 UTC   DEBUG:  writing stats file "pg_stat_tmp/db_0.stat"
2021-02-17 15:56:41.466 UTC   DEBUG:  InitPostgres
2021-02-17 15:56:41.466 UTC   DEBUG:  my backend ID is 3
2021-02-17 15:56:41.466 UTC   DEBUG:  StartTransaction(1) name: unnamed; blockState: DEFAULT; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:41.467 UTC   DEBUG:  CommitTransaction(1) name: unnamed; blockState: STARTED; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:41.467 UTC   DEBUG:  autovacuum: processing database "airflow_db"
2021-02-17 15:56:41.467 UTC   DEBUG:  received inquiry for database 16385
2021-02-17 15:56:41.467 UTC   DEBUG:  writing stats file "pg_stat_tmp/global.stat"
2021-02-17 15:56:41.467 UTC   DEBUG:  writing stats file "pg_stat_tmp/db_16385.stat"
2021-02-17 15:56:41.467 UTC   DEBUG:  writing stats file "pg_stat_tmp/db_0.stat"
2021-02-17 15:56:41.477 UTC   DEBUG:  StartTransaction(1) name: unnamed; blockState: DEFAULT; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:41.477 UTC   DEBUG:  pg_statistic: vac: 0 (threshold 134), anl: 0 (threshold 92)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_type: vac: 0 (threshold 131), anl: 0 (threshold 91)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_authid: vac: 0 (threshold 52), anl: 1 (threshold 51)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_attribute: vac: 0 (threshold 633), anl: 0 (threshold 341)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_proc: vac: 0 (threshold 642), anl: 0 (threshold 346)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_class: vac: 0 (threshold 129), anl: 0 (threshold 90)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_index: vac: 0 (threshold 82), anl: 0 (threshold 66)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_operator: vac: 0 (threshold 204), anl: 0 (threshold 127)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_opclass: vac: 0 (threshold 76), anl: 0 (threshold 63)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_am: vac: 0 (threshold 51), anl: 0 (threshold 51)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_amop: vac: 0 (threshold 193), anl: 0 (threshold 122)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_amproc: vac: 0 (threshold 139), anl: 0 (threshold 95)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_rewrite: vac: 0 (threshold 75), anl: 0 (threshold 63)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_cast: vac: 0 (threshold 93), anl: 0 (threshold 72)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_namespace: vac: 0 (threshold 51), anl: 0 (threshold 51)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_database: vac: 0 (threshold 50), anl: 1 (threshold 50)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_tablespace: vac: 0 (threshold 50), anl: 0 (threshold 50)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_shdepend: vac: 0 (threshold 52), anl: 1 (threshold 51)
2021-02-17 15:56:41.478 UTC   DEBUG:  pg_toast_2618: vac: 0 (threshold 100), anl: 0 (threshold 75)
2021-02-17 15:56:41.478 UTC   DEBUG:  CommitTransaction(1) name: unnamed; blockState: STARTED; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:41.478 UTC   DEBUG:  shmem_exit(0): 1 before_shmem_exit callbacks to make
2021-02-17 15:56:41.478 UTC   DEBUG:  shmem_exit(0): 7 on_shmem_exit callbacks to make
2021-02-17 15:56:41.478 UTC   DEBUG:  proc_exit(0): 2 callbacks to make
2021-02-17 15:56:41.478 UTC   DEBUG:  exit(0)
2021-02-17 15:56:41.478 UTC   DEBUG:  shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:56:41.478 UTC   DEBUG:  shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:56:41.478 UTC   DEBUG:  proc_exit(-1): 0 callbacks to make
2021-02-17 15:56:41.478 UTC   DEBUG:  reaping dead processes
2021-02-17 15:56:41.478 UTC   DEBUG:  server process (PID 29567) exited with exit code 0
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG:  shmem_exit(0): 0 before_shmem_exit callbacks to make
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG:  shmem_exit(0): 0 on_shmem_exit callbacks to make
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG:  proc_exit(0): 1 callbacks to make
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG:  exit(0)
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG:  shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG:  shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:56:47.646 UTC [unknown] [unknown] DEBUG:  proc_exit(-1): 0 callbacks to make
2021-02-17 15:56:47.647 UTC   DEBUG:  forked new backend, pid=29613 socket=9
2021-02-17 15:56:47.647 UTC   DEBUG:  reaping dead processes
2021-02-17 15:56:47.647 UTC   DEBUG:  server process (PID 29413) exited with exit code 0
2021-02-17 15:56:47.647 UTC [unknown] [unknown] LOG:  connection received: host=127.0.0.1 port=45008
2021-02-17 15:56:47.647 UTC [unknown] airflow_user DEBUG:  postgres child[29613]: starting with (
2021-02-17 15:56:47.647 UTC [unknown] airflow_user DEBUG:   postgres
2021-02-17 15:56:47.647 UTC [unknown] airflow_user DEBUG:  )
2021-02-17 15:56:47.647 UTC [unknown] airflow_user DEBUG:  InitPostgres
2021-02-17 15:56:47.648 UTC [unknown] airflow_user DEBUG:  my backend ID is 3
2021-02-17 15:56:47.648 UTC [unknown] airflow_user DEBUG:  StartTransaction(1) name: unnamed; blockState: DEFAULT; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:47.648 UTC [unknown] airflow_user LOG:  connection authorized: user=airflow_user database=airflow_db application_name=psql
2021-02-17 15:56:47.649 UTC psql airflow_user DEBUG:  CommitTransaction(1) name: unnamed; blockState: STARTED; state: INPROGRESS, xid/subid/cid: 0/1/0
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG:  shmem_exit(0): 1 before_shmem_exit callbacks to make
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG:  shmem_exit(0): 6 on_shmem_exit callbacks to make
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG:  proc_exit(0): 4 callbacks to make
2021-02-17 15:56:50.746 UTC psql airflow_user LOG:  disconnection: session time: 0:00:03.098 user=airflow_user database=airflow_db host=127.0.0.1 port=45008
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG:  exit(0)
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG:  shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG:  shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:56:50.746 UTC psql airflow_user DEBUG:  proc_exit(-1): 0 callbacks to make
2021-02-17 15:56:50.746 UTC   DEBUG:  reaping dead processes
2021-02-17 15:56:50.746 UTC   DEBUG:  server process (PID 29613) exited with exit code 0
2021-02-17 15:57:00.668 UTC   DEBUG:  postmaster received signal 2
2021-02-17 15:57:00.668 UTC   LOG:  received fast shutdown request
2021-02-17 15:57:00.668 UTC   LOG:  aborting any active transactions
2021-02-17 15:57:00.668 UTC   DEBUG:  sending signal 15 to process 26962
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(0): 1 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  autovacuum launcher shutting down
2021-02-17 15:57:00.668 UTC   DEBUG:  logical replication launcher shutting down
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(0): 5 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(1): 2 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(0): 1 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  proc_exit(0): 2 callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(0): 6 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  exit(0)
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(1): 6 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  proc_exit(0): 2 callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  proc_exit(1): 2 callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  exit(0)
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(0): 1 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  exit(1)
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(0): 5 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  proc_exit(0): 2 callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  exit(0)
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.668 UTC   DEBUG:  proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.669 UTC   DEBUG:  reaping dead processes
2021-02-17 15:57:00.669 UTC   LOG:  background worker "logical replication launcher" (PID 26962) exited with exit code 1
2021-02-17 15:57:00.669 UTC   DEBUG:  reaping dead processes
2021-02-17 15:57:00.669 UTC   LOG:  shutting down
2021-02-17 15:57:00.670 UTC   DEBUG:  performing replication slot checkpoint
2021-02-17 15:57:00.672 UTC   DEBUG:  attempting to remove WAL segments older than log file 000000000000000000000000
2021-02-17 15:57:00.672 UTC   DEBUG:  SlruScanDirectory invoking callback on pg_subtrans/0000
2021-02-17 15:57:00.672 UTC   DEBUG:  shmem_exit(0): 1 before_shmem_exit callbacks to make
2021-02-17 15:57:00.672 UTC   DEBUG:  shmem_exit(0): 5 on_shmem_exit callbacks to make
2021-02-17 15:57:00.672 UTC   DEBUG:  proc_exit(0): 2 callbacks to make
2021-02-17 15:57:00.672 UTC   DEBUG:  exit(0)
2021-02-17 15:57:00.672 UTC   DEBUG:  shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.672 UTC   DEBUG:  shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.672 UTC   DEBUG:  proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.672 UTC   DEBUG:  reaping dead processes
2021-02-17 15:57:00.672 UTC   DEBUG:  writing stats file "pg_stat/global.stat"
2021-02-17 15:57:00.672 UTC   DEBUG:  writing stats file "pg_stat/db_16385.stat"
2021-02-17 15:57:00.673 UTC   DEBUG:  removing temporary stats file "pg_stat_tmp/db_16385.stat"
2021-02-17 15:57:00.673 UTC   DEBUG:  writing stats file "pg_stat/db_12738.stat"
2021-02-17 15:57:00.673 UTC   DEBUG:  removing temporary stats file "pg_stat_tmp/db_12738.stat"
2021-02-17 15:57:00.673 UTC   DEBUG:  writing stats file "pg_stat/db_0.stat"
2021-02-17 15:57:00.673 UTC   DEBUG:  removing temporary stats file "pg_stat_tmp/db_0.stat"
2021-02-17 15:57:00.673 UTC   DEBUG:  shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.673 UTC   DEBUG:  shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.673 UTC   DEBUG:  proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.673 UTC   DEBUG:  reaping dead processes
2021-02-17 15:57:00.673 UTC   DEBUG:  shmem_exit(0): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.673 UTC   DEBUG:  shmem_exit(0): 5 on_shmem_exit callbacks to make
2021-02-17 15:57:00.673 UTC   DEBUG:  cleaning up dynamic shared memory control segment with ID 521889379
2021-02-17 15:57:00.676 UTC   DEBUG:  proc_exit(0): 2 callbacks to make
2021-02-17 15:57:00.676 UTC   LOG:  database system is shut down
2021-02-17 15:57:00.676 UTC   DEBUG:  exit(0)
2021-02-17 15:57:00.676 UTC   DEBUG:  shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.676 UTC   DEBUG:  shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.676 UTC   DEBUG:  proc_exit(-1): 0 callbacks to make
2021-02-17 15:57:00.677 UTC   DEBUG:  logger shutting down
2021-02-17 15:57:00.677 UTC   DEBUG:  shmem_exit(0): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.677 UTC   DEBUG:  shmem_exit(0): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.677 UTC   DEBUG:  proc_exit(0): 0 callbacks to make
2021-02-17 15:57:00.677 UTC   DEBUG:  exit(0)
2021-02-17 15:57:00.677 UTC   DEBUG:  shmem_exit(-1): 0 before_shmem_exit callbacks to make
2021-02-17 15:57:00.677 UTC   DEBUG:  shmem_exit(-1): 0 on_shmem_exit callbacks to make
2021-02-17 15:57:00.677 UTC   DEBUG:  proc_exit(-1): 0 callbacks to make

我注意到的一件事是,在“收到连接”消息和下一步之间,大约需要 11 秒。我已经在我的个人桌面上测试过这个(不工作)并且连接几乎是即时的,这似乎表明这是某种环境问题。

如果有人有任何见解,我很乐意听到!我已经被屏蔽了很长时间了。

非常感谢!

1 个答案:

答案 0 :(得分:1)

问题解决了。 Anaconda 的 PostgreSQL 包有问题,导致每次连接尝试失败一次,然后在第二次尝试时成功。不知道到底发生了什么,但我们使用良好的旧版本从源代码构建了 PostgreSQL,它的工作原理非常棒。

大家加油