elasticsearch logstash重试失败的操作,响应代码:429 锿:,elasticsearch,logstash,elasticsearch,Logstash" /> elasticsearch logstash重试失败的操作,响应代码:429 锿:,elasticsearch,logstash,elasticsearch,Logstash" />

elasticsearch logstash重试失败的操作,响应代码:429 锿:

elasticsearch logstash重试失败的操作,响应代码:429 锿:,elasticsearch,logstash,elasticsearch,Logstash,节点1-节点10。10台机器 cpu:32核 jvm:60G 日志存储: logstash-node1,logstash-node2 cpu:32核 jvm 20G 每台机器上安装4个logstsh应用程序 数据源:18w/s-20w/s 数据格式: "index" : { "number_of_shards" : 8, "number_of_replicas" : 0, "refresh_interval" : "30s" }

节点1-节点10。10台机器

cpu:32核

jvm:60G

日志存储: logstash-node1,logstash-node2

cpu:32核

jvm 20G

每台机器上安装4个logstsh应用程序

数据源:18w/s-20w/s

数据格式:

    "index" : {
        "number_of_shards" : 8, 
        "number_of_replicas" : 0,
        "refresh_interval" : "30s"
    }

      "filed1" : {
              "type": "integer"
      },
      "filed2" : {
              "type": "integer"
      },
      "filed3" : {
              "type": "long"
      },
      "filed4" : { 
              "type": "text"
      },
      "filed5":{
        "type": "text"
      },
      "filed6":
      {
        "type": "text"
      },
      "fild7":{
        "type": "date"
      },
      "field8":{
        "type": "geo_point"
      }
    }
logstash.yml //这默认为主机的CPU核心数

管道工人:20人

//每个输出插件实例应该使用多少工人

管道生产工人:20人

//在发送到筛选器+工作者之前,要从输入中检索多少事件 管道.批次.尺寸:9000

logstash kafka.conf 问: 现在速度为800w-900w/min

速度慢,

和logstash始终输出错误:

[2017-05-11T09:27:57,327][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@2393d3d9 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@63898a2e[Running, pool size = 32, active threads = 32, queued tasks = 50, completed tasks = 27163960]]"})

如何提高测井速度。厚度。

es版本:5.2 logstash版本:5.2
[2017-05-11T09:27:57,327][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@2393d3d9 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@63898a2e[Running, pool size = 32, active threads = 32, queued tasks = 50, completed tasks = 27163960]]"})