由于以下错误，Logstash停止处理：（SystemExit）exit

我们正在尝试在Elasticsearch中分别索引Nginx访问和错误日志。为此，我们创建了Filbeat和Logstash配置，如下所示

下面是我们的

/etc/filebeat/filebeat.yml

配置

filebeat.inputs:
- type: log
  paths:
    - /var/log/nginx/*access*.log
  exclude_files: ['\.gz$']
  exclude_lines: ['*ELB-HealthChecker*']
  fields:
    log_type: type1 
- type: log
  paths:
    - /var/log/nginx/*error*.log
  exclude_files: ['\.gz$']
  exclude_lines: ['*ELB-HealthChecker*']
  fields:
    log_type: type2

output.logstash:
  hosts: ["10.227.XXX.XXX:5400"]

我们的logstash文件

/etc/logstash/conf.d/logstash nginx es.conf

config如下

input {
    beats {
        port => 5400
    }
}

filter {
  if ([fields][log_type] == "type1") {
    grok {
      match => [ "message" , "%{NGINXACCESS}+%{GREEDYDATA:extra_fields}"]
      overwrite => [ "message" ]
    }
    mutate {
      convert => ["response", "integer"]
      convert => ["bytes", "integer"]
      convert => ["responsetime", "float"]
    }
    geoip {
      source => "clientip"
      target => "geoip"
      add_tag => [ "nginx-geoip" ]
    }
    date {
      match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
      remove_field => [ "timestamp" ]
    }
    useragent {
      source => "user_agent"
    }
  } else {
      grok {
        match => [ "message" , "(?<timestamp>%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY} %{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER:threadid}\: \*%{NUMBER:connectionid} %{GREEDYDATA:message}, client: %{IP:client}, server: %{GREEDYDATA:server}, request: "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion}))"(, upstream: "%{GREEDYDATA:upstream}")?, host: "%{DATA:host}"(, referrer: "%{GREEDYDATA:referrer}")?"]
        overwrite => [ "message" ]
      }
      mutate {
        convert => ["response", "integer"]
        convert => ["bytes", "integer"]
        convert => ["responsetime", "float"]
      }
      geoip {
        source => "clientip"
        target => "geoip"
        add_tag => [ "nginx-geoip" ]
      }
      date {
        match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
        remove_field => [ "timestamp" ]
      }
      useragent {
        source => "user_agent"
      }
    }
}

output {
  if ([fields][log_type] == "type1") {
    amazon_es {
      hosts => ["vpc-XXXX-XXXX.ap-southeast-1.es.amazonaws.com"]
      region => "ap-southeast-1"
      aws_access_key_id => 'XXXX'
      aws_secret_access_key => 'XXXX'
      index => "nginx-access-logs-%{+YYYY.MM.dd}"
    }
} else {
    amazon_es {
      hosts => ["vpc-XXXX-XXXX.ap-southeast-1.es.amazonaws.com"]
      region => "ap-southeast-1"
      aws_access_key_id => 'XXXX'
      aws_secret_access_key => 'XXXX'
      index => "nginx-error-logs-%{+YYYY.MM.dd}"
    }
  }
    stdout { 
      codec => rubydebug 
    }
}

输入{
击败{
端口=>5400
}
}
滤器{
如果（[字段][日志类型]=“类型1”）{
格罗克{
match=>[“message”，“%{NGINXACCESS}+%{greedydydata:extra_fields}”]
覆盖=>[“消息”]
}
变异{
转换=>[“响应”，“整数”]
转换=>[“字节”，“整数”]
转换=>[“响应时间”，“浮动”]
}
geoip{
source=>“clientip”
target=>“geoip”
添加标签=>[“nginx geoip”]
}
日期{
匹配=>[“时间戳”，“dd/MMM/YYYY:HH:mm:ss Z”]
删除_字段=>[“时间戳”]
}
用户代理{
source=>“用户\代理”
}
}否则{
格罗克{
match=>[“message”，“（？%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY}%{TIME}）\[%{LOGLEVEL:severity}\]%{POSINT:pid}\\\\{NUMBER:threadid}:\*%{NUMBER:connectionid}%{greedData:message client}，客户端：%{IP:client}，服务器：{greedData:server}，请求：“（？：%:{WORD WORD verb:verb:threadid}，verb:threadid}版本：”{HTTP:%}，上游：{？，主机：“%{DATA:host}”（，引用方：“%{greedyddata:referer}”）？”]
覆盖=>[“消息”]
}
变异{
转换=>[“响应”，“整数”]
转换=>[“字节”，“整数”]
转换=>[“响应时间”，“浮动”]
}
geoip{
source=>“clientip”
target=>“geoip”
添加标签=>[“nginx geoip”]
}
日期{
匹配=>[“时间戳”，“dd/MMM/YYYY:HH:mm:ss Z”]
删除_字段=>[“时间戳”]
}
用户代理{
source=>“用户\代理”
}
}
}
输出{
如果（[字段][日志类型]=“类型1”）{
亚马逊{
主机=>[“vpc XXXX XXXX.ap-southerast-1.es.amazonaws.com”]
地区=>“ap-东南-1”
aws\u访问\u密钥\u id=>“XXXX”
aws_secret_access_key=>“XXXX”
index=>“nginx访问日志-%{+YYYY.MM.dd}”
}
}否则{
亚马逊{
主机=>[“vpc XXXX XXXX.ap-southerast-1.es.amazonaws.com”]
地区=>“ap-东南-1”
aws\u访问\u密钥\u id=>“XXXX”
aws_secret_access_key=>“XXXX”
index=>“nginx错误日志-%{+YYYY.MM.dd}”
}
}
标准输出{
编解码器=>rubydebug
}
}

我们在启动logstash时收到以下错误

[2020-10-12T06:05:39,183][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.9.2", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 25.265-b01 on 1.8.0_265-b01 +indy +jit [linux-x86_64]"}
[2020-10-12T06:05:39,861][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-10-12T06:05:41,454][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"{\", \",\", \"]\" at line 32, column 263 (byte 918) after filter {\n  if ([fields][log_type] == \"type1\") {\n    grok {\n      match => [ \"message\" , \"%{NGINXACCESS}+%{GREEDYDATA:extra_fields}\"]\n      overwrite => [ \"message\" ]\n    }\n    mutate {\n      convert => [\"response\", \"integer\"]\n      convert => [\"bytes\", \"integer\"]\n      convert => [\"responsetime\", \"float\"]\n    }\n    geoip {\n      source => \"clientip\"\n      target => \"geoip\"\n      add_tag => [ \"nginx-geoip\" ]\n    }\n    date {\n      match => [ \"timestamp\" , \"dd/MMM/YYYY:HH:mm:ss Z\" ]\n      remove_field => [ \"timestamp\" ]\n    }\n    useragent {\n      source => \"user_agent\"\n    }\n  } else {\n      grok {\n        match => [ \"message\" , \"(?<timestamp>%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY} %{TIME}) \\[%{LOGLEVEL:severity}\\] %{POSINT:pid}#%{NUMBER:threadid}\\: \\*%{NUMBER:connectionid} %{GREEDYDATA:message}, client: %{IP:client}, server: %{GREEDYDATA:server}, request: \"", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:183:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:44:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:357:in `block in converge_state'"]}
[2020-10-12T06:05:41,795][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2020-10-12T06:05:46,685][INFO ][logstash.runner          ] Logstash shut down.
[2020-10-12T06:05:46,706][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

[2020-10-12T06:05:39183][INFO][logstash.runner]启动logstash{“logstash.version”=>“7.9.2”，“jruby.version”=>“jruby 9.2.13.0（2.5.7）2020-08-03 9a89c94bcc OpenJDK 64位服务器VM 25.265-b01 on 1.8.0265-b01+indy+jit[linux-x8664]”
[2020-10-12T06:05:39861][WARN][logstash.config.source.multilocal]忽略“pipelines.yml”文件，因为指定了模块或命令行选项
[2020-10-12T06:05:41454][ERROR][logstash.agent]未能执行操作{:action=>logstash:：PipelineAction:：Create/pipeline\u id:main，：exception=>“logstash:：ConfigurationError”，：message=>“应为[\\t\\r\\n]，\“{\”，\”，\“，\”，\“，\”，“]\”，“\]”在第32行，第263列（字节918），如果（[fields][log\u type]=“type1\”）{\n grok{\n match=>[\'message\'，\'{NGINXACCESS}+%{greedyddata:extra\u fields}\']\n overwrite=>[\'message\']\n}\n mutate{\n convert=>[\'response\'，\'integer\']\n convert=>[\'bytes\'，\'integer\']\n convert=>[\'responsetime\'，\'float\'\n}\n geoip{\n源=>'\“\n target=>\“geoip\”\n add\u tag=>[\“nginx geoip\”]\n}\n date{\n match=>[\“timestamp\”，\“dd/MMM/yyyyy:HH:mm:ss Z\”]\n remove\u field=>[\“timestamp\”]\n}\n}\n useragent{\n source source=>“user\u agent\”\n}\n}\n}\n}其他{\n grok{\n match=>[\”message message{MONTHDAY}%{TIME}）\\[%{LOGLEVEL:severity}\\\]%{POSINT:pid}{NUMBER:threadid}\\\\\*%{NUMBER:connectionid}%{GREEDYDATA:message}，客户端：%{IP:client}，服务器：%{greeddata:server}，请求：\，：backtrace=>[“/logusr/share/stash/logstash-core/lib/logstash/compiler.rb 32:in`compileorg/logstash/execution/AbstractPipelineExt.java:183:in`initialize'，“org/logstash/execution/JavaBasePipelineExt.java:69:in`initialize'，/usr/share/logstash/logstash-core/lib/logstash/pipeline_-action/create.rb:52:in`execute'”/usr/share/logstash/logstash core/lib/logstash/agent.rb:357:in`block in converge_state''}
[2020-10-12T06:05:41795][INFO][logstash.agent]已成功启动logstash API端点{:port=>9600}
[2020-10-12T06:05:46685][INFO][logstash.runner]logstash关闭。
[2020-10-12T06:05:46706][ERROR][org.logstash.logstash]java.lang.IllegalStateException:logstash因错误而停止处理：（SystemExit）退出

似乎存在一些格式问题。请帮助解决问题

==============================================更新===================================

对于所有希望为nginx访问和错误日志寻找健壮的grok过滤器的人……请尝试以下过滤器模式

访问日志-

%{IPORHOST:remote\u-ip}-%{DATA:user\u-name}\[%{HTTPDATE:Access\u-time}\]\“{WORD:http\u-method}%{URIPATHPARAM:url}http/%{NUMBER:http\u-version\}{NUMBER:response\u-code}%{NUMBER:body\u-sent\u-bytes\\\\\\\\\{SPACE referer}\\\\\{DATA:agent\\\\\\\%%NUMBER:duration req头数据：{req:req\\\{body:}\“resp_头：\”%{DATA:resp_头}\“resp_体：\”%{GREEDYDATA:resp_体}\”

错误日志-

（？%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY}%{TIME}）\[%{LOGLEVEL:severity}\]%{POSINT:pid}{number}
grok {
        match => [ "message" , "(?<timestamp>%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY} %{TIME})\[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER:threadid}\: \*%{NUMBER:connectionid} %{GREEDYDATA:message}, client: %{IP:client}, server: %{GREEDYDATA:server}, request: \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion}))\"(, upstream: \"%{GREEDYDATA:upstream}\")?, host: \"%{DATA:host}\"(, referrer: \"%{GREEDYDATA:referrer}\")?"]
        overwrite => [ "message" ]
      }