elasticsearch Elasticsearch在logstash错误[HTTP输出失败]中按查询更新遇到非2xx HTTP代码400,elasticsearch,logstash,logstash-configuration,elasticsearch,Logstash,Logstash Configuration" /> elasticsearch Elasticsearch在logstash错误[HTTP输出失败]中按查询更新遇到非2xx HTTP代码400,elasticsearch,logstash,logstash-configuration,elasticsearch,Logstash,Logstash Configuration" />

elasticsearch Elasticsearch在logstash错误[HTTP输出失败]中按查询更新遇到非2xx HTTP代码400

elasticsearch Elasticsearch在logstash错误[HTTP输出失败]中按查询更新遇到非2xx HTTP代码400,elasticsearch,logstash,logstash-configuration,elasticsearch,Logstash,Logstash Configuration,正在工作\u通过\u查询调用更新\u- POST /s1test-demo7/_update_by_query { "script": { "source": "ctx._source.externaldata = params.externaldata", "lang": "painless", "params": {

正在工作\u通过\u查询调用更新\u-

POST /s1test-demo7/_update_by_query
{
  "script": {
      "source": "ctx._source.externaldata = params.externaldata",
      "lang": "painless",
      "params": {
        "externaldata":{
          "field1": "1",
          "field2": "abc"  
        }
      }
    },
  "query": {
    "bool": {
      "must": [
        { "match": { "h.req-id": "Test9GrpA"} },
        { "match": { "h.process-code": "DemoS99"} }
      ]
    }
  }
}
这个API实际上添加了“externaldata”字段和两个内部字段,作为与现有文档匹配的查询中的嵌套json。
我需要logstash管道的帮助来完成同样的工作-

input {
    file {
        id => "updatedata"
        path => "D:/p3-test1.json"
        start_position => "beginning"
        sincedb_path => "D:/sdb/p3-test1.sdb"
    }
}
filter {
    json {
        source => "message"
    }
    mutate {
        add_field => {
            "[script][lang]" => "painless"
            "[script][source]" => "ctx._source.externaldata = params.externaldata"
            "[script][params][externaldata][field1]" => "%{field1}"
            "[script][params][externaldata][field2]" => "%{field2}"
            "[query][bool][must][match][h.req-id]" => "%{req-id}"
            "[query][bool][must][match][h.process-code]" => "%{process-code}"
        }
    }
}
output {
    stdout {
        codec => rubydebug
    }
    http {
        url => "http://localhost:9200/s1test-demo7/_update_by_query"
        headers => { "Authorization" => "Basic ZWxhc3RpYzplbGFzdGlj" }
        http_method => "post"
        format => "json"
    }
}
logstash管道必须包括身份验证,否则我会得到HTTP错误401。但我不确定这里的语法是否正确。文档和elasticsearch论坛帖子中的http输出插件标题有所不同。
日志存储输出-

[2021-05-29T20:14:13,226][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
{
            "host" => "mypc",
        "@version" => "1",
          "req-id" => "Test9GrpA",
          "field2" => "default",
      "@timestamp" => 2021-05-29T14:44:13.521Z,
          "field1" => "1",
           "query" => {
        "bool" => {
            "must" => {
                "match" => {
                    "h.process-code" => "DemoS99",
                          "h.req-id" => "Test9GrpA"
                }
            }
        }
    },
         "message" => "{\"req-id\":\"Test9GrpA\",\"process-code\":\"DemoS99\",\"field1\":\"1\",\"field2\":\"default\"}\r",
    "process-code" => "DemoS99",
          "script" => {
          "lang" => "painless",
        "params" => {
            "externaldata" => {
                "field2" => "default",
                "field1" => "1"
            }
        },
        "source" => "ctx._source.externaldata = params.externaldata"
    },
            "path" => "D:/p3-test1.json"
}
[2021-05-29T20:14:14,268][ERROR][logstash.outputs.http    ][main][2bc46338fec26d73c819043dac159f1d12397fefc800c84c7d2e0f0d16b278c2] [HTTP Output Failure] Encountered non-2xx HTTP code 400 {:response_code=>400, :url=>"http://localhost:9200/s1test-demo7/_update_by_query", :event=>#<LogStash::Event:0x2badd231>}
[2021-05-29T20:14:13226][INFO][logstash.agent]正在运行的管道{:count=>1,:正在运行的管道=>[:main],:未运行的管道=>[]
{
“主机”=>“mypc”,
“@version”=>“1”,
“请求id”=>“Test9GrpA”,
“字段2”=>“默认值”,
“@timestamp”=>2021-05-29T14:44:13.521Z,
“字段1”=>“1”,
“查询”=>{
“bool”=>{
“必须”=>{
“匹配”=>{
“h.process-code”=>“DemoS99”,
“h.req-id”=>“Test9GrpA”
}
}
}
},
“消息”=>“{\“req id\”:“Test9GrpA\”,“过程代码\”:“DemoS99\”,“field1\”:“1\”,“field2\”:“default\”}\r”,
“过程代码”=>“演示99”,
“脚本”=>{
“朗”=>“无痛”,
“参数”=>{
“外部数据”=>{
“字段2”=>“默认值”,
“字段1”=>“1”
}
},
“source”=>“ctx.\u source.externaldata=params.externaldata”
},
“路径”=>“D:/p3-test1.json”
}
[2021-05-29T20:14:14268][ERROR][logstash.outputs.http][main][2BC46338FEC26D73C819043DAC159F12397FEFC800C84C7D2E0F0D16B278C2][http输出失败]遇到非2xx http代码400{:响应_代码=>400,:url=>http://localhost:9200/s1test-demo7/_update_by_query“,:event=>#}
请提供帮助。

可以将脚本部分转换为“
http
”输出插件,而不是在过滤器部分使用“
无痛
”脚本

output {
    stdout {
        codec => rubydebug
    }
    if "externallogs" in [tags] {
        http {
            url => "http://localhost:9200/s1test-demo7/_update_by_query"
            headers => { "Authorization" => "Basic ZWxhc3RpYzplbGFzdGlj" }
            http_method => "post"
            format => "message"
            content_type => "application/json"
            message => '{  
                    "script": {
                      "source": "ctx._source.externaldata = params.externaldata",
                      "lang": "painless",
                      "params": {
                        "externaldata":{
                          "field1": "%{field1}",
                          "field2": "%{field2}"  
                        }
                      }
                    },
                  "query": {
                    "bool": {
                      "must": [
                        { "match": { "h.req-id": "%{req-id}"} },
                        { "match": { "h.process-code": "%{process-code}"} }
                      ]
                    }
                  }
                }'
        }
    }
}