elasticsearch,logstash,logstash-grok,Regex,elasticsearch,Logstash,Logstash Grok" /> elasticsearch,logstash,logstash-grok,Regex,elasticsearch,Logstash,Logstash Grok" />

Regex Logstash grok过滤器apache模式

Regex Logstash grok过滤器apache模式,regex,elasticsearch,logstash,logstash-grok,Regex,elasticsearch,Logstash,Logstash Grok,这是一个Apache Tomcat日志示例: portal.portal.some.thing.int:8443 13.233.220.113 - - [09/Sep/2019:00:08:02 +0200] "GET /en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&face

这是一个Apache Tomcat日志示例:

portal.portal.some.thing.int:8443 13.233.220.113 - - [09/Sep/2019:00:08:02 +0200] "GET /en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1" 230 334734 6261 - - 35S64857F6860FDFC0F60B5B47A97E18
10.235.350.103 94.62.15.157, 10.435.230.101,10.134.046.2
我想捕获以下变量

2019年9月9日:00:08:02+0200

/en/搜索结果?p_p_id=portal2….

35S64857F6860FDFC0F60B5B47A97E18

你能帮我吗?我只想索引这些,而删除其他的,有可能吗?谢谢

使用此grok模式:

%{GREEDYDATA:field1} %{IP:ip1} - - \[%{GREEDYDATA:date}] \"%{WORD:method} %{GREEDYDATA:request}" %{WORD:numbers} %{WORD:numbers} %{WORD:numbers} - - %{WORD:last_parameter}
输入:

portal.portal.some.thing.int:8443 13.233.220.113 - - [09/Sep/2019:00:08:02 +0200] "GET /en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1" 230 334734 6261 - - 35S64857F6860FDFC0F60B5B47A97E18
10.235.350.103 94.62.15.157, 10.435.230.101,10.134.046.2
{
  "field1": [
    [
      "portal.portal.some.thing.int:8443"
    ]
  ],
  "ip1": [
    [
      "13.233.220.113"
    ]
  ],
  "IPV6": [
    [
      null
    ]
  ],
  "IPV4": [
    [
      "13.233.220.113"
    ]
  ],
  "date": [
    [
      "09/Sep/2019:00:08:02 +0200"
    ]
  ],
  "method": [
    [
      "GET"
    ]
  ],
  "request": [
    [
      "/en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1"
    ]
  ],
  "numbers": [
    [
      "230",
      "334734",
      "6261"
    ]
  ],
  "last_parameter": [
    [
      "35S64857F6860FDFC0F60B5B47A97E18"
    ]
  ]
}
输出:

portal.portal.some.thing.int:8443 13.233.220.113 - - [09/Sep/2019:00:08:02 +0200] "GET /en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1" 230 334734 6261 - - 35S64857F6860FDFC0F60B5B47A97E18
10.235.350.103 94.62.15.157, 10.435.230.101,10.134.046.2
{
  "field1": [
    [
      "portal.portal.some.thing.int:8443"
    ]
  ],
  "ip1": [
    [
      "13.233.220.113"
    ]
  ],
  "IPV6": [
    [
      null
    ]
  ],
  "IPV4": [
    [
      "13.233.220.113"
    ]
  ],
  "date": [
    [
      "09/Sep/2019:00:08:02 +0200"
    ]
  ],
  "method": [
    [
      "GET"
    ]
  ],
  "request": [
    [
      "/en/search-results?p_p_id=portal201_WAR_portal201_INSTANCE_q8EzsBteHybf&p_p_lifecycle=1&p_p_state=normal&queryText=Poll&facet.collection=AΜLex%2CAMsom%2CAMss%2WebPage%2SummariesOfSomething&startRow=1&resultsPerPage=10&SEARCH_TYPE=SIMPLE HTTP/1.1"
    ]
  ],
  "numbers": [
    [
      "230",
      "334734",
      "6261"
    ]
  ],
  "last_parameter": [
    [
      "35S64857F6860FDFC0F60B5B47A97E18"
    ]
  ]
}
您需要的字段是:

  • 日期
  • 请求
  • 最后一个参数
您可以使用in mutate filter删除其他字段