Database PostgreSQL逻辑复制应该使用多少CPU/带宽?

Database PostgreSQL逻辑复制应该使用多少CPU/带宽?,database,postgresql,Database,Postgresql,我使用PostgreSQL 11设置逻辑复制,如下所示: 一切都很顺利,经过测试,更改得以复制 然而,一个月后。。。变化似乎不会被复制,Postgres似乎使用了大量的CPU和带宽 2vCPU/4GB DigitalOcean服务器上的平均负载约为2.5 带宽约为1MB/s 现在这个服务器和数据库上基本上没有活动 这引发了一些问题,比如: 对于具有逻辑流复制的非活动数据库来说,使用这么多资源是否正常 有没有关于复制为何停止的想法?(更改主服务器上的记录不再影响复制副本) 是否有一些用于监视和

我使用PostgreSQL 11设置逻辑复制,如下所示:

一切都很顺利,经过测试,更改得以复制

然而,一个月后。。。变化似乎不会被复制,Postgres似乎使用了大量的CPU和带宽

  • 2vCPU/4GB DigitalOcean服务器上的平均负载约为2.5
  • 带宽约为1MB/s
  • 现在这个服务器和数据库上基本上没有活动
这引发了一些问题,比如:

  • 对于具有逻辑流复制的非活动数据库来说,使用这么多资源是否正常
  • 有没有关于复制为何停止的想法?(更改主服务器上的记录不再影响复制副本)
  • 是否有一些用于监视和查看复制状态的专业提示
  • Postgres服务器的主日志中充满了以下消息:

    2019-04-22 06:26:16.986 UTC [20371] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21198
    2019-04-22 06:26:16.986 UTC [20371] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.010 UTC [20372] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC211D0
    2019-04-22 06:26:17.010 UTC [20372] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.055 UTC [20373] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21208
    2019-04-22 06:26:17.055 UTC [20373] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.078 UTC [20374] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21240
    2019-04-22 06:26:17.078 UTC [20374] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.114 UTC [20375] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21278
    2019-04-22 06:26:17.114 UTC [20375] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.154 UTC [20376] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC212B0
    2019-04-22 06:26:17.154 UTC [20376] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.186 UTC [20377] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC212E8
    2019-04-22 06:26:17.186 UTC [20377] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.229 UTC [20378] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21320
    2019-04-22 06:26:17.229 UTC [20378] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.235 UTC [20378] replica_user@server_prod LOG:  could not send data to client: Connection reset by peer
    2019-04-22 06:26:17.235 UTC [20378] replica_user@server_prod STATEMENT:  COPY public.class_registrations TO STDOUT
    2019-04-22 06:26:17.235 UTC [20378] replica_user@server_prod FATAL:  connection to client lost
    2019-04-22 06:26:17.235 UTC [20378] replica_user@server_prod STATEMENT:  COPY public.class_registrations TO STDOUT
    2019-04-22 06:26:17.259 UTC [20379] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21358
    2019-04-22 06:26:17.259 UTC [20379] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:21.327 UTC [20418] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21390
    2019-04-22 06:26:21.327 UTC [20418] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:21.341 UTC [20419] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC213C8
    2019-04-22 06:26:21.341 UTC [20419] replica_user@server_prod DETAIL:  There are no running transactions.
    
    2019-04-21 06:26:07.619 UTC [2967] LOG:  logical replication table synchronization worker for subscription "replica_subscription", table "messages" has started
    2019-04-21 06:26:07.645 UTC [2966] ERROR:  duplicate key value violates unique constraint "account_locations_pkey"
    2019-04-21 06:26:07.645 UTC [2966] DETAIL:  Key (id)=(1) already exists.
    2019-04-21 06:26:07.645 UTC [2966] CONTEXT:  COPY account_locations, line 1
    2019-04-21 06:26:07.648 UTC [16353] LOG:  background worker "logical replication worker" (PID 2966) exited with exit code 1
    2019-04-21 06:26:07.652 UTC [2968] LOG:  logical replication table synchronization worker for subscription "replica_subscription", table "user_photos" has started
    2019-04-21 06:26:07.663 UTC [2967] ERROR:  duplicate key value violates unique constraint "messages_pkey"
    2019-04-21 06:26:07.663 UTC [2967] DETAIL:  Key (id)=(1) already exists.
    2019-04-21 06:26:07.663 UTC [2967] CONTEXT:  COPY messages, line 1
    
    副本服务器中充满了以下类型的消息:

    2019-04-22 06:26:16.986 UTC [20371] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21198
    2019-04-22 06:26:16.986 UTC [20371] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.010 UTC [20372] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC211D0
    2019-04-22 06:26:17.010 UTC [20372] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.055 UTC [20373] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21208
    2019-04-22 06:26:17.055 UTC [20373] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.078 UTC [20374] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21240
    2019-04-22 06:26:17.078 UTC [20374] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.114 UTC [20375] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21278
    2019-04-22 06:26:17.114 UTC [20375] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.154 UTC [20376] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC212B0
    2019-04-22 06:26:17.154 UTC [20376] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.186 UTC [20377] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC212E8
    2019-04-22 06:26:17.186 UTC [20377] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.229 UTC [20378] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21320
    2019-04-22 06:26:17.229 UTC [20378] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:17.235 UTC [20378] replica_user@server_prod LOG:  could not send data to client: Connection reset by peer
    2019-04-22 06:26:17.235 UTC [20378] replica_user@server_prod STATEMENT:  COPY public.class_registrations TO STDOUT
    2019-04-22 06:26:17.235 UTC [20378] replica_user@server_prod FATAL:  connection to client lost
    2019-04-22 06:26:17.235 UTC [20378] replica_user@server_prod STATEMENT:  COPY public.class_registrations TO STDOUT
    2019-04-22 06:26:17.259 UTC [20379] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21358
    2019-04-22 06:26:17.259 UTC [20379] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:21.327 UTC [20418] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC21390
    2019-04-22 06:26:21.327 UTC [20418] replica_user@server_prod DETAIL:  There are no running transactions.
    2019-04-22 06:26:21.341 UTC [20419] replica_user@server_prod LOG:  logical decoding found consistent point at 0/1EC213C8
    2019-04-22 06:26:21.341 UTC [20419] replica_user@server_prod DETAIL:  There are no running transactions.
    
    2019-04-21 06:26:07.619 UTC [2967] LOG:  logical replication table synchronization worker for subscription "replica_subscription", table "messages" has started
    2019-04-21 06:26:07.645 UTC [2966] ERROR:  duplicate key value violates unique constraint "account_locations_pkey"
    2019-04-21 06:26:07.645 UTC [2966] DETAIL:  Key (id)=(1) already exists.
    2019-04-21 06:26:07.645 UTC [2966] CONTEXT:  COPY account_locations, line 1
    2019-04-21 06:26:07.648 UTC [16353] LOG:  background worker "logical replication worker" (PID 2966) exited with exit code 1
    2019-04-21 06:26:07.652 UTC [2968] LOG:  logical replication table synchronization worker for subscription "replica_subscription", table "user_photos" has started
    2019-04-21 06:26:07.663 UTC [2967] ERROR:  duplicate key value violates unique constraint "messages_pkey"
    2019-04-21 06:26:07.663 UTC [2967] DETAIL:  Key (id)=(1) already exists.
    2019-04-21 06:26:07.663 UTC [2967] CONTEXT:  COPY messages, line 1
    
    这是过去6小时内的平均负载(您可以在我删除副本服务器上的订阅服务器时看到)

    这里是带宽:

    这里还有一个
    iftop
    的监测结果,只需10-15秒:


    根据Laurenz的建议查看日志后,我最初加载的数据似乎没有在主ID上对所有表进行正确的顺序。(不知道这是怎么发生的)

    为了解决复制问题,我执行了以下操作:

  • 已从副本服务器中删除订阅
  • 删除所有表格
  • 重新加载所有表-仅架构(无数据)
  • 再次创建订阅
  • 这导致所有数据都被同步,一切又恢复正常。我通过更新数据并在副本服务器中看到数据更新来确认


    高CPU负载和带宽似乎是在出现复制错误时出现的,Postgres只是尽其所能反复尝试。

    查看PostgreSQL日志文件。很好的调用--它确实充满了消息--我将它们添加到了问题中。备用服务器呢?啊,答对了!我想这是一个主要的关键问题,当我复制初始数据库作为起点时。。。按键未重置为最新值。