Logstash 无法正确使用grok日期匹配

Logstash 无法正确使用grok日期匹配,logstash,logstash-grok,grok,Logstash,Logstash Grok,Grok,我有这个消息 2016/02/22 08:40:10[错误]2127#0:*193公开() “/etc/nginx/nginx/html/static cdn.arte.tv/resize prepod/nqa5owndknadsxe0mpemd5mcua=/940x530/smart/default/prog\u img/img\u APIOS/051000/051700/051757-001_1137283\u 32\u 202.jpg” 失败(2:没有这样的文件或目录),客户端:192.1

我有这个消息

2016/02/22 08:40:10[错误]2127#0:*193公开() “/etc/nginx/nginx/html/static cdn.arte.tv/resize prepod/nqa5owndknadsxe0mpemd5mcua=/940x530/smart/default/prog\u img/img\u APIOS/051000/051700/051757-001_1137283\u 32\u 202.jpg” 失败(2:没有这样的文件或目录),客户端:192.168.144.95,服务器: api.magritte.arte.tv,请求:“获取 /静态cdn.arte.tv/resize prepod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog\u img/img\u APIOS/051000/051700/051757-001\u 1137283\u 32\u 202.jpg HTTP/1.1,“主持人:“api.magritte.arte.tv”,参考人: “”

我这样分析它

grok {
      match => { "message" => "(?<timestamp>%{YEAR}/%{MONTHNUM2}/%{MONTHDAY} %{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER:tid}:( \*%{NUMBER:cid})? %{GREEDYDATA:errormessage}(?:, client: (?<client>%{IP}|%{HOSTNAME}))(?:, server: %{IPORHOST:server})(?:, request: %{QS:request})?(?:, upstream: \"%{URI:upstream}\")?(?:, host: %{QS:host})?(?:, referrer: \"%{URI:referrer}\")?"}
    }
    date {
      match => [ "timestamp" , "YYYY/MM/dd HH:mm:ss" ]
    }
grok{
{MONTHDAY}{{MONTHDAY}}{{{{{MONTHDAY}}{{{{{时间}{{{{{{信息”消息“{{{信息”消息“{{{消息”{{{消息“{{消息”{{{{{消息“{{{{{年年}}{年}}}{{{时间}{{{时间时间}{{{时间时间}}{{{{{{{{{{时间时间时间}}}{{{{{{{时间}}}}}}}{{{{{{{{{{{{{{{{{{{{{{{{{{{门门门门门级:严重程度:严重程度:严重程度:门门门门门门门门门门门门门门门门门门门门门门门门门门门市:严重程度:严重程度:::}}}}}}}}}}QS:host})?(?:,推荐人:\”%{URI:referer}\”?}
}
日期{
匹配=>[“时间戳”,“YYYY/MM/dd HH:MM:ss”]
}
当新消息到达时,会发生以下行为

  • 发送给兔子的消息:好的
  • 兔子的留言:好的
  • logstash读取消息时出现问题
  • “原因”=>“未能分析[时间戳]”, 由“=>{”类型“=>”非法参数引起的“\u异常”,“原因”=>”无效 格式:\“2016/02/22 08:40:10\”在\“/02/22处格式不正确 08:40:10\“”},:level=>:warn}

    但我不知道我的错误在哪里。使用一切似乎都可以

    日志库中的完整日志为

    {:timestamp=>"2016-02-22T08:43:29.968000+0100", :message=>"Failed action. ", :status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2016.02.22", :_type=>"nginx_error", :_routing=>nil}, #<LogStash::Event:0x75f8f9a0 @metadata_accessors=#<LogStash::Util::Accessors:0x402f1514 @store={}, @lut={}>, @cancelled=false, @data={"message"=>"2016/02/22 08:40:10 [error] 2127#0: *193 open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory), client: 192.168.144.95, server: api.magritte.arte.tv, request: \"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\", host: \"api.magritte.arte.tv\", referrer: \"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos\"", "@version"=>"1", "@timestamp"=>"2016-02-22T07:40:10.000Z", "path"=>"/var/log/nginx/api.magritte.arte.tv_error.log", "host"=>["magritte.arte.tv", "\"api.magritte.arte.tv\""], "type"=>"nginx_error", "application"=>"api", "timestamp"=>"2016/02/22 08:40:10", "severity"=>"error", "pid"=>2127, "tid"=>0, "cid"=>193, "errormessage"=>"open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory)", "client"=>"192.168.144.95", "server"=>"api.magritte.arte.tv", "request"=>"\"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\"", "referrer"=>"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos"}, @metadata={}, @accessors=#<LogStash::Util::Accessors:0x27ca0e3f @store={"message"=>"2016/02/22 08:40:10 [error] 2127#0: *193 open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory), client: 192.168.144.95, server: api.magritte.arte.tv, request: \"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\", host: \"api.magritte.arte.tv\", referrer: \"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos\"", "@version"=>"1", "@timestamp"=>"2016-02-22T07:40:10.000Z", "path"=>"/var/log/nginx/api.magritte.arte.tv_error.log", "host"=>["magritte.arte.tv", "\"api.magritte.arte.tv\""], "type"=>"nginx_error", "application"=>"api", "timestamp"=>"2016/02/22 08:40:10", "severity"=>"error", "pid"=>2127, "tid"=>0, "cid"=>193, "errormessage"=>"open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory)", "client"=>"192.168.144.95", "server"=>"api.magritte.arte.tv", "request"=>"\"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\"", "referrer"=>"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos"}, @lut={"type"=>[{"message"=>"2016/02/22 08:40:10 [error] 2127#0: *193 open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory), client: 192.168.144.95, server: api.magritte.arte.tv, request: \"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\", host: \"api.magritte.arte.tv\", referrer: \"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos\"", "@version"=>"1", "@timestamp"=>"2016-02-22T07:40:10.000Z", "path"=>"/var/log/nginx/api.magritte.arte.tv_error.log", "host"=>["magritte.arte.tv", "\"api.magritte.arte.tv\""], "type"=>"nginx_error", "application"=>"api", "timestamp"=>"2016/02/22 08:40:10", "severity"=>"error", "pid"=>2127, "tid"=>0, "cid"=>193, "errormessage"=>"open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory)", "client"=>"192.168.144.95", "server"=>"api.magritte.arte.tv", "request"=>"\"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\"", "referrer"=>"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos"}, "type"], "[type]"=>[{"message"=>"2016/02/22 08:40:10 [error] 2127#0: *193 open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory), client: 192.168.144.95, server: api.magritte.arte.tv, request: \"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\", host: \"api.magritte.arte.tv\", referrer: \"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos\"", "@version"=>"1", "@timestamp"=>"2016-02-22T07:40:10.000Z", "path"=>"/var/log/nginx/api.magritte.arte.tv_error.log", "host"=>["magritte.arte.tv", "\"api.magritte.arte.tv\""], "type"=>"nginx_error", "application"=>"api", "timestamp"=>"2016/02/22 08:40:10", "severity"=>"error", "pid"=>2127, "tid"=>0, "cid"=>193, "errormessage"=>"open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory)", "client"=>"192.168.144.95", "server"=>"api.magritte.arte.tv", "request"=>"\"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\"", "referrer"=>"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos"}, "type"]}>>], :response=>{"create"=>{"_index"=>"logstash-2016.02.22", "_type"=>"nginx_error", "_id"=>"AVMH7uSoo1ZDC2Pzezhl", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [timestamp]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"2016/02/22 08:40:10\" is malformed at \"/02/22 08:40:10\""}}}}, :level=>:warn}
    
    {timestamp=>“2016-02-22T08:43:29.968000+0100”,:message=>“操作失败”,:status=>400,:action=>[“索引”,{:\u id=>nil,:\u index=>“logstash-2016.02.22”,:\u type=>“nginx\u错误”,:\u routing=>nil},\/etc/nginx/nginx/html/static cdn.arte.tv/resize prepod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog\u img/img\u APIOS/051000/051700/051757-001\u 1137283\u 32\u 202.jpg\“失败(2:没有此类文件或目录),客户端:192.168.144.95,服务器:api.magritte.arte.tv,请求:”GET/static cdn.arte.tv/resize prepod/nqa5owndknadsxe0mpemd5mcua=/940x530/smart/default/prog\u img/img\u APIOS/051000/051700/051757-001\u 1137283\u 32\u 202.jpg HTTP/1.1\,主机:\“api.magritte.arte.tv\”,参考者:“https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos\“”、“@version”=>“1”、“@timestamp”=>“2016-02-22T07:40:10.000Z”,“路径”=>“/var/log/nginx/api.magritte.arte.tv_error.log”,“主机”=>[“magritte.arte.tv”,“api.magritte.arte.tv\”,“类型”=>“nginx_错误”,“应用程序”=>“api”,“时间戳”=>“2016/02/22 08:40:10”,“严重性”=>“错误”,“pid”=>2127,“tid”=>0,“cid”=>193”错误消息“=>“打开”)\“/etc/nginx/nginx/html/static cdn.arte.tv/resize prepod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog\u img/img\u APIOS/051000/051700/051757-001\u 1137283\u 32\u 202.jpg\”失败(2:无此类文件或目录)”,“客户端”=>“192.168.144.95”,“服务器”=>“api.magrite.arte.tv”,“请求”=>”GET/static cdn.arte.tv/resize prepod/nqa5owndknadsxe0Mpemd5Mcua=/940x530/smart/default/prog_img/img_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\,“referer=>”https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos“},@metadata={},@accessors=#”2016/02/22 08:40:10[error]2127#0:*193 open()\“/etc/nginx/nginx/html/static cdn.arte.tv/resize prepod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog\u img/img\u APIOS/051000/051700/051757-001\u 1137283\u 32\u 202.jpg\”失败(2:没有此类文件或目录),客户端:192.168.144.95,服务器:api.magritte.arte.tv,请求:\”GET/static cdn.arte.tv/resize prepod/nqa5owndknadsxe0mpemd5mcua=/940x530/smart/default/prog\u img/img\u APIOS/051000/051700/051757-001\u 1137283\u 32\u 202.jpg HTTP/1.1\,主机:\“api.magritte.arte.tv\”,参考者:“https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos\“”、“@version”=>“1”、“@timestamp”=>“2016-02-22T07:40:10.000Z”,“路径”=>“/var/log/nginx/api.magritte.arte.tv_error.log”,“主机”=>[“magritte.arte.tv”,“api.magritte.arte.tv\”,“类型”=>“nginx_错误”,“应用程序”=>“api”,“时间戳”=>“2016/02/22 08:40:10”,“严重性”=>“错误”,“pid”=>2127,“tid”=>0,“cid”=>193”错误消息“=>“打开”)\“/etc/nginx/nginx/html/static cdn.arte.tv/resize prepod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog\u img/img\u APIOS/051000/051700/051757-001\u 1137283\u 32\u 202.jpg\”失败(2:无此类文件或目录)”,“客户端”=>“192.168.144.95”,“服务器”=>“api.magrite.arte.tv”,“请求”=>”GET/static cdn.arte.tv/resize prepod/nqa5owndknadsxe0Mpemd5Mcua=/940x530/smart/default/prog_img/img_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\,“referer=>”https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos“},@lut={”type“=>[{”message“=>”2016/02/22 08:40:10[error]2127#0:*193 open()\“/etc/nginx/nginx/html/static cdn.arte.tv/resize prepod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog\u img/img\u APIOS/051000/051700/051757-001\u 1137283\u 32\u 202.jpg\”失败(2:没有此类文件或目录),客户端:192.168.144.95,服务器:api.magritte.arte.tv,请求:\”GET/static cdn.arte.tv/resize prepod/nqa5owndknadsxe0mpemd5mcua=/940x530/smart/default/prog\u img/img\u APIOS/051000/051700/051757-001\u 1137283\u 32\u 202.jpg HTTP/1.1\,主机:\“api.magritte.arte.tv\”,参考者:“https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos\“”、“@version”=>“1”、“@timestamp”=>“2016-02-22T07:40:10.000Z”,“路径”=>“/var/log/nginx/api.magritte.arte.tv_error.log”,“主机”=>[“magritte.arte.tv”,“api.magritte.arte.tv\”,“类型”=>“nginx_错误”,“应用程序”=>“api”,“时间戳”=>“2016/02/22 08:40:10”,“严重性”=>“错误”,“pid”=>2127,“tid”=>0,“cid”=>193”错误消息“=>“打开”)\“/etc/nginx/nginx/html/static-cdn.arte.tv/resize-prepod/nQa5oWnNDknADSxe0mPEMd
    
    grok {
           match => ["message","%{DATESTAMP:timestamp}" ]
    }
    
    date {
      match => [ "timestamp" , "yyyy/MM/dd HH:mm:ss" ]
    }
    
     grok {
          match => { "message" => "(?<timestamp>%{YEAR}/%{MONTHNUM2}/%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}) \[%{LOGLEVEL:severity}\] %{POSINT:p_id}#%{NUMBER:t_id}:( \*%{NUMBER:c_id})? %{GREEDYDATA:errormessage}(?:, client: (?<client>%{IP}|%{HOSTNAME}))(?:, server: %{IPORHOST:server})(?:, request: %{QS:request})?(?:, upstream: %{QS:upstream})?(?:, host: %{QS:vhost})?(?:, referrer: \"%{URI:referrer}\")?"}
        }
      date {
      match => [ "timestamp" , "yyyy/MM/dd HH:mm:ss" ]
    }