elasticsearch Elasticsearch在logstash错误[HTTP输出失败]中按查询更新遇到非2xx HTTP代码400
正在工作\u通过\u查询调用更新\u-elasticsearch Elasticsearch在logstash错误[HTTP输出失败]中按查询更新遇到非2xx HTTP代码400,elasticsearch,logstash,logstash-configuration,elasticsearch,Logstash,Logstash Configuration,正在工作\u通过\u查询调用更新\u- POST /s1test-demo7/_update_by_query { "script": { "source": "ctx._source.externaldata = params.externaldata", "lang": "painless", "params": {
POST /s1test-demo7/_update_by_query
{
"script": {
"source": "ctx._source.externaldata = params.externaldata",
"lang": "painless",
"params": {
"externaldata":{
"field1": "1",
"field2": "abc"
}
}
},
"query": {
"bool": {
"must": [
{ "match": { "h.req-id": "Test9GrpA"} },
{ "match": { "h.process-code": "DemoS99"} }
]
}
}
}
这个API实际上添加了“externaldata”字段和两个内部字段,作为与现有文档匹配的查询中的嵌套json。
我需要logstash管道的帮助来完成同样的工作-
input {
file {
id => "updatedata"
path => "D:/p3-test1.json"
start_position => "beginning"
sincedb_path => "D:/sdb/p3-test1.sdb"
}
}
filter {
json {
source => "message"
}
mutate {
add_field => {
"[script][lang]" => "painless"
"[script][source]" => "ctx._source.externaldata = params.externaldata"
"[script][params][externaldata][field1]" => "%{field1}"
"[script][params][externaldata][field2]" => "%{field2}"
"[query][bool][must][match][h.req-id]" => "%{req-id}"
"[query][bool][must][match][h.process-code]" => "%{process-code}"
}
}
}
output {
stdout {
codec => rubydebug
}
http {
url => "http://localhost:9200/s1test-demo7/_update_by_query"
headers => { "Authorization" => "Basic ZWxhc3RpYzplbGFzdGlj" }
http_method => "post"
format => "json"
}
}
logstash管道必须包括身份验证,否则我会得到HTTP错误401。但我不确定这里的语法是否正确。文档和elasticsearch论坛帖子中的http输出插件标题有所不同。
日志存储输出-
[2021-05-29T20:14:13,226][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
{
"host" => "mypc",
"@version" => "1",
"req-id" => "Test9GrpA",
"field2" => "default",
"@timestamp" => 2021-05-29T14:44:13.521Z,
"field1" => "1",
"query" => {
"bool" => {
"must" => {
"match" => {
"h.process-code" => "DemoS99",
"h.req-id" => "Test9GrpA"
}
}
}
},
"message" => "{\"req-id\":\"Test9GrpA\",\"process-code\":\"DemoS99\",\"field1\":\"1\",\"field2\":\"default\"}\r",
"process-code" => "DemoS99",
"script" => {
"lang" => "painless",
"params" => {
"externaldata" => {
"field2" => "default",
"field1" => "1"
}
},
"source" => "ctx._source.externaldata = params.externaldata"
},
"path" => "D:/p3-test1.json"
}
[2021-05-29T20:14:14,268][ERROR][logstash.outputs.http ][main][2bc46338fec26d73c819043dac159f1d12397fefc800c84c7d2e0f0d16b278c2] [HTTP Output Failure] Encountered non-2xx HTTP code 400 {:response_code=>400, :url=>"http://localhost:9200/s1test-demo7/_update_by_query", :event=>#<LogStash::Event:0x2badd231>}
[2021-05-29T20:14:13226][INFO][logstash.agent]正在运行的管道{:count=>1,:正在运行的管道=>[:main],:未运行的管道=>[]
{
“主机”=>“mypc”,
“@version”=>“1”,
“请求id”=>“Test9GrpA”,
“字段2”=>“默认值”,
“@timestamp”=>2021-05-29T14:44:13.521Z,
“字段1”=>“1”,
“查询”=>{
“bool”=>{
“必须”=>{
“匹配”=>{
“h.process-code”=>“DemoS99”,
“h.req-id”=>“Test9GrpA”
}
}
}
},
“消息”=>“{\“req id\”:“Test9GrpA\”,“过程代码\”:“DemoS99\”,“field1\”:“1\”,“field2\”:“default\”}\r”,
“过程代码”=>“演示99”,
“脚本”=>{
“朗”=>“无痛”,
“参数”=>{
“外部数据”=>{
“字段2”=>“默认值”,
“字段1”=>“1”
}
},
“source”=>“ctx.\u source.externaldata=params.externaldata”
},
“路径”=>“D:/p3-test1.json”
}
[2021-05-29T20:14:14268][ERROR][logstash.outputs.http][main][2BC46338FEC26D73C819043DAC159F12397FEFC800C84C7D2E0F0D16B278C2][http输出失败]遇到非2xx http代码400{:响应_代码=>400,:url=>http://localhost:9200/s1test-demo7/_update_by_query“,:event=>#}
请提供帮助。可以将脚本部分转换为“http
”输出插件,而不是在过滤器部分使用“无痛
”脚本
output {
stdout {
codec => rubydebug
}
if "externallogs" in [tags] {
http {
url => "http://localhost:9200/s1test-demo7/_update_by_query"
headers => { "Authorization" => "Basic ZWxhc3RpYzplbGFzdGlj" }
http_method => "post"
format => "message"
content_type => "application/json"
message => '{
"script": {
"source": "ctx._source.externaldata = params.externaldata",
"lang": "painless",
"params": {
"externaldata":{
"field1": "%{field1}",
"field2": "%{field2}"
}
}
},
"query": {
"bool": {
"must": [
{ "match": { "h.req-id": "%{req-id}"} },
{ "match": { "h.process-code": "%{process-code}"} }
]
}
}
}'
}
}
}