Linux serf：查询队列深度警告无休止的计数，没有关于错误源的文档

从昨天起，我们开始在作业运行之间接收到此错误，该错误无休止地向上计数（到目前为止，在手动重新启动之前已达到1245）

我们正在运行Dkron作业调度程序，ETCD作为后端

示例错误：

信息[2019-02-14T10:03:39+02:00]2019/02/14 10:03:39[警告]serf:查询队列深度：218

似乎没有任何文件或帮助我理解这一点，我已经能够找到。如果有人知道这是什么原因，或者有更多关于它的信息，我将非常感激

下面我将添加一份更详细的日志副本，以防其中任何一份可能有所帮助。再次感谢你！（抱歉，如果我没有提供足够的信息，我不确定哪些信息是重要的，因为我不知道这个错误的来源）

INFO[2019-02-14T10:03:49+02:00] 2019/02/14 10:03:49 [WARN] serf: Query queue depth: 218 
INFO[2019-02-14T10:03:50+02:00] 2019/02/14 10:03:50 [WARN] serf: Query queue depth: 218 
INFO[2019-02-14T10:03:51+02:00] 2019/02/14 10:03:51 [WARN] serf: Query queue depth: 218 
INFO[2019-02-14T10:03:52+02:00] 2019/02/14 10:03:52 [WARN] serf: Query queue depth: 218 
INFO[2019-02-14T10:03:53+02:00] 2019/02/14 10:03:53 [WARN] serf: Query queue depth: 218 
INFO[2019-02-14T10:03:54+02:00] 2019/02/14 10:03:54 [WARN] serf: Query queue depth: 218 
INFO[2019-02-14T10:03:55+02:00] 2019/02/14 10:03:55 [WARN] serf: Query queue depth: 218 
INFO[2019-02-14T10:03:56+02:00] 2019/02/14 10:03:56 [WARN] serf: Query queue depth: 218 
INFO[2019-02-14T10:03:57+02:00] 2019/02/14 10:03:57 [WARN] serf: Query queue depth: 218 
INFO[2019-02-14T10:03:58+02:00] 2019/02/14 10:03:58 [WARN] serf: Query queue depth: 218 
INFO[2019-02-14T10:03:59+02:00] 2019/02/14 10:03:59 [WARN] serf: Query queue depth: 218 
DEBU[2019-02-14T10:04:00+02:00] scheduler: Run job                            job=60minute_jobs node=Dkron_Main schedule="0 4 * * * *"
DEBU[2019-02-14T10:04:00+02:00] scheduler: Run job                            job=2minute_jobs node=Dkron_Main schedule="0 */2 * * * *"
DEBU[2019-02-14T10:04:00+02:00] store: Retrieved job from datastore           job=2minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] agent: Filtered nodes to run: []              node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] agent: Filtered tags to run: map[]            node=Dkron_Main
INFO[2019-02-14T10:04:00+02:00] agent: Sending query                          job_name=2minute_jobs node=Dkron_Main query="run:job"
DEBU[2019-02-14T10:04:00+02:00] agent: Sending query                          job_name=2minute_jobs json="{\"execution\":{\"job_name\":\"2minute_jobs\",\"started_at\":\"0001-01-01T00:00:00Z\",\"finished_at\":\"0001-01-01T00:00:00Z\",\"group\":1550131440007370602,\"attempt\":1},\"rpc_addr\":\"127.0.0.1:6868\"}" node=Dkron_Main query="run:job"
INFO[2019-02-14T10:04:00+02:00] agent: Received event                         event="query: run:job" node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] agent: Running job                            at=219 node=Dkron_Main payload="{\"execution\":{\"job_name\":\"2minute_jobs\",\"started_at\":\"0001-01-01T00:00:00Z\",\"finished_at\":\"0001-01-01T00:00:00Z\",\"group\":1550131440007370602,\"attempt\":1},\"rpc_addr\":\"127.0.0.1:6868\"}" query="run:job"
INFO[2019-02-14T10:04:00+02:00] agent: Starting job                           job=2minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] scheduler: Run job                            job=1minute_jobs node=Dkron_Main schedule="0 * * * * *"
INFO[2019-02-14T10:04:00+02:00] 2019/02/14 10:04:00 [DEBUG] serf: messageQueryResponseType: Dkron_Main 
DEBU[2019-02-14T10:04:00+02:00] agent: Received ack                           from=Dkron_Main node=Dkron_Main query="run:job"
DEBU[2019-02-14T10:04:00+02:00] store: Retrieved job from datastore           job=60minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] agent: Filtered nodes to run: []              node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] agent: Filtered tags to run: map[]            node=Dkron_Main
INFO[2019-02-14T10:04:00+02:00] agent: Sending query                          job_name=60minute_jobs node=Dkron_Main query="run:job"
DEBU[2019-02-14T10:04:00+02:00] agent: Sending query                          job_name=60minute_jobs json="{\"execution\":{\"job_name\":\"60minute_jobs\",\"started_at\":\"0001-01-01T00:00:00Z\",\"finished_at\":\"0001-01-01T00:00:00Z\",\"group\":1550131440007318183,\"attempt\":1},\"rpc_addr\":\"127.0.0.1:6868\"}" node=Dkron_Main query="run:job"
INFO[2019-02-14T10:04:00+02:00] 2019/02/14 10:04:00 [DEBUG] serf: messageQueryResponseType: Dkron_Main 
DEBU[2019-02-14T10:04:00+02:00] agent: Received ack                           from=Dkron_Main node=Dkron_Main query="run:job"
DEBU[2019-02-14T10:04:00+02:00] store: Retrieved job from datastore           job=1minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] agent: Filtered nodes to run: []              node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] agent: Filtered tags to run: map[]            node=Dkron_Main
INFO[2019-02-14T10:04:00+02:00] agent: Sending query                          job_name=1minute_jobs node=Dkron_Main query="run:job"
DEBU[2019-02-14T10:04:00+02:00] agent: Sending query                          job_name=1minute_jobs json="{\"execution\":{\"job_name\":\"1minute_jobs\",\"started_at\":\"0001-01-01T00:00:00Z\",\"finished_at\":\"0001-01-01T00:00:00Z\",\"group\":1550131440027758329,\"attempt\":1},\"rpc_addr\":\"127.0.0.1:6868\"}" node=Dkron_Main query="run:job"
INFO[2019-02-14T10:04:00+02:00] 2019/02/14 10:04:00 [DEBUG] serf: messageQueryResponseType: Dkron_Main 
DEBU[2019-02-14T10:04:00+02:00] agent: Received ack                           from=Dkron_Main node=Dkron_Main query="run:job"
DEBU[2019-02-14T10:04:00+02:00] grpc: Received GetJob                         job=2minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] store: Retrieved job from datastore           job=2minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] agent: GetJob by RPC                          job=2minute_jobs node=Dkron_Main
INFO[2019-02-14T10:04:00+02:00] agent: Received event                         event="query: run:job" node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] agent: Running job                            at=220 node=Dkron_Main payload="{\"execution\":{\"job_name\":\"60minute_jobs\",\"started_at\":\"0001-01-01T00:00:00Z\",\"finished_at\":\"0001-01-01T00:00:00Z\",\"group\":1550131440007318183,\"attempt\":1},\"rpc_addr\":\"127.0.0.1:6868\"}" query="run:job"
INFO[2019-02-14T10:04:00+02:00] agent: Starting job                           job=60minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] invoke: calling executor plugin               node=Dkron_Main plugin=shell
INFO[2019-02-14T10:04:00+02:00] 2019/02/14 10:04:00 [DEBUG] serf: messageQueryResponseType: Dkron_Main 
DEBU[2019-02-14T10:04:00+02:00] agent: Received response                      from=Dkron_Main node=Dkron_Main query="run:job" response="{\"job_name\":\"2minute_jobs\",\"started_at\":\"2019-02-14T10:04:00.126925316+02:00\",\"finished_at\":\"0001-01-01T00:00:00Z\",\"node_name\":\"Dkron_Main\",\"group\":1550131440007370602,\"attempt\":1}"
DEBU[2019-02-14T10:04:00+02:00] store: Setting key                            execution=1550131440126925316-Dkron_Main job=2minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] shell: going to run bash /home/benemenadmin/dkron_jobs/bash/2minute_jobs.sh  node=Dkron_Main subsystem_name=plugins.dkron-executor-shell
DEBU[2019-02-14T10:04:00+02:00] store: to detele key                          execution=1550119440072343171-Dkron_Main job=2minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] grpc: Received GetJob                         job=60minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] store: Retrieved job from datastore           job=60minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] agent: GetJob by RPC                          job=60minute_jobs node=Dkron_Main
INFO[2019-02-14T10:04:00+02:00] agent: Received event                         event="query: run:job" node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] agent: Running job                            at=221 node=Dkron_Main payload="{\"execution\":{\"job_name\":\"1minute_jobs\",\"started_at\":\"0001-01-01T00:00:00Z\",\"finished_at\":\"0001-01-01T00:00:00Z\",\"group\":1550131440027758329,\"attempt\":1},\"rpc_addr\":\"127.0.0.1:6868\"}" query="run:job"
INFO[2019-02-14T10:04:00+02:00] agent: Starting job                           job=1minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] invoke: calling executor plugin               node=Dkron_Main plugin=shell
INFO[2019-02-14T10:04:00+02:00] 2019/02/14 10:04:00 [DEBUG] serf: messageQueryResponseType: Dkron_Main 
DEBU[2019-02-14T10:04:00+02:00] agent: Received response                      from=Dkron_Main node=Dkron_Main query="run:job" response="{\"job_name\":\"60minute_jobs\",\"started_at\":\"2019-02-14T10:04:00.202107834+02:00\",\"finished_at\":\"0001-01-01T00:00:00Z\",\"node_name\":\"Dkron_Main\",\"group\":1550131440007318183,\"attempt\":1}"
DEBU[2019-02-14T10:04:00+02:00] store: Setting key                            execution=1550131440202107834-Dkron_Main job=60minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] shell: going to run bash /home/benemenadmin/dkron_jobs/bash/60minute_jobs.sh  node=Dkron_Main subsystem_name=plugins.dkron-executor-shell
DEBU[2019-02-14T10:04:00+02:00] store: to detele key                          execution=1549757040048767013-Dkron_Main job=60minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] grpc: Received GetJob                         job=1minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] store: Retrieved job from datastore           job=1minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] agent: GetJob by RPC                          job=1minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] invoke: calling executor plugin               node=Dkron_Main plugin=shell
DEBU[2019-02-14T10:04:00+02:00] shell: going to run bash /home/benemenadmin/dkron_jobs/bash/1minute_jobs.sh  node=Dkron_Main subsystem_name=plugins.dkron-executor-shell
INFO[2019-02-14T10:04:00+02:00] 2019/02/14 10:04:00 [DEBUG] serf: messageQueryResponseType: Dkron_Main 
DEBU[2019-02-14T10:04:00+02:00] agent: Received response                      from=Dkron_Main node=Dkron_Main query="run:job" response="{\"job_name\":\"1minute_jobs\",\"started_at\":\"2019-02-14T10:04:00.316814676+02:00\",\"finished_at\":\"0001-01-01T00:00:00Z\",\"node_name\":\"Dkron_Main\",\"group\":1550131440027758329,\"attempt\":1}"
DEBU[2019-02-14T10:04:00+02:00] store: Setting key                            execution=1550131440316814676-Dkron_Main job=1minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:00+02:00] store: to detele key                          execution=1550125380180647982-Dkron_Main job=1minute_jobs node=Dkron_Main
INFO[2019-02-14T10:04:00+02:00] 2019/02/14 10:04:00 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:01+02:00] 2019/02/14 10:04:01 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:02+02:00] 2019/02/14 10:04:02 [WARN] serf: Query queue depth: 221 
DEBU[2019-02-14T10:04:03+02:00] agent: Done receiving acks and responses      node=Dkron_Main query="run:job"
DEBU[2019-02-14T10:04:03+02:00] agent: Done receiving acks and responses      node=Dkron_Main query="run:job"
DEBU[2019-02-14T10:04:03+02:00] agent: Done receiving acks and responses      node=Dkron_Main query="run:job"
INFO[2019-02-14T10:04:03+02:00] 2019/02/14 10:04:03 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:04+02:00] 2019/02/14 10:04:04 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:05+02:00] 2019/02/14 10:04:05 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:06+02:00] 2019/02/14 10:04:06 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:07+02:00] 2019/02/14 10:04:07 [WARN] serf: Query queue depth: 221 
DEBU[2019-02-14T10:04:07+02:00] shell: Command output                         node=Dkron_Main subsystem_name=plugins.dkron-executor-shell
DEBU[2019-02-14T10:04:07+02:00]                                               node=Dkron_Main subsystem_name=plugins.dkron-executor-shell
DEBU[2019-02-14T10:04:07+02:00] 2Minute_Jobs complete                         node=Dkron_Main subsystem_name=plugins.dkron-executor-shell
DEBU[2019-02-14T10:04:07+02:00]                                               node=Dkron_Main subsystem_name=plugins.dkron-executor-shell
DEBU[2019-02-14T10:04:07+02:00] grpc: Received execution done                 group=1550131440007370602 job=2minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:07+02:00] store: Retrieved job from datastore           job=2minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:07+02:00] store: Setting key                            execution=1550131440126925316-Dkron_Main job=2minute_jobs node=Dkron_Main
DEBU[2019-02-14T10:04:08+02:00] grpc: from: Dkron_Main                        node=Dkron_Main
INFO[2019-02-14T10:04:08+02:00] 2019/02/14 10:04:08 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:09+02:00] 2019/02/14 10:04:09 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:10+02:00] 2019/02/14 10:04:10 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:11+02:00] 2019/02/14 10:04:11 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:12+02:00] 2019/02/14 10:04:12 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:13+02:00] 2019/02/14 10:04:13 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:14+02:00] 2019/02/14 10:04:14 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:15+02:00] 2019/02/14 10:04:15 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:16+02:00] 2019/02/14 10:04:16 [WARN] serf: Query queue depth: 221 
INFO[2019-02-14T10:04:17+02:00] 2019/02/14 10:04:17 [WARN] serf: Query queue depth: 221