Warning: file_get_contents(/data/phpspider/zhask/data//catemap/4/regex/16.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/8/logging/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Regex 具有多主机ip的Fluentd apache日志格式_Regex_Logging - Fatal编程技术网

Regex 具有多主机ip的Fluentd apache日志格式

Regex 具有多主机ip的Fluentd apache日志格式,regex,logging,Regex,Logging,我对fluend日志解析器有一个小问题。我有一个varnish服务器,在该服务器上设置了X-Forwarded-For参数,以包含http请求所经过的所有主机堆栈的ip列表。我使用它在varnishncsa日志中获取信息。以下是日志的示例: "192.168.79.16, 192.22.10.22, 10.2.2.22 - - [13/Aug/2015:09:50:45 +0000] \"GET http://poc.mydomain.com/panier/payment/payline?not

我对fluend日志解析器有一个小问题。我有一个varnish服务器,在该服务器上设置了X-Forwarded-For参数,以包含http请求所经过的所有主机堆栈的ip列表。我使用它在varnishncsa日志中获取信息。以下是日志的示例:

"192.168.79.16, 192.22.10.22, 10.2.2.22 - - [13/Aug/2015:09:50:45 +0000] \"GET http://poc.mydomain.com/panier/payment/payline?notificationType=WEBTRS&token=1KB01BwKWdUhVj1222301439454223514 HTTP/1.1\" 401 0 \"-\" \"Java/1.8.0_45\""
在订单方面,我希望在fluentd上聚合这些日志。然后,由于NCSA日志使用apache格式,我使用apache2flentd格式进行输入解析,如下配置:

<source>
  type tail
  format apache2
  path /var/log/varnish/varnishncsa.log
  pos_file /var/log/td-agent/tmp/access.log.pos
  tag "apache2.varnish.mydomain.com.access"
</source>
这不匹配:

"192.168.79.16, 192.22.10.22, 10.2.2.22 - - [13/Aug/2015:09:50:45 +0000] \"GET http://poc.mydomain.com/panier/payment/payline?notificationType=WEBTRS&token=1KB01BwKWdUhVj1222301439454223514 HTTP/1.1\" 401 0 \"-\" \"Java/1.8.0_45\""
apache2 fluentd正则表达式是:

^(?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$
我试图找到并用文本发送正确的regx,但还没有找到

我试过了,但没用

 <source>
      type tail
      format format /^(?<host>\,*[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$/ 
      time_format %d/%b/%Y:%H:%M:%S %z
      path /var/log/varnish/varnishncsa.log
      pos_file /var/log/td-agent/tmp/access.log.pos
      tag "apache2.varnish.mydomain.com.access"
    </source>

型尾
格式格式/^(?\,*[^]*)[^]*(?[^]*)\[(?[^\]*)\]“(?\S+(?[^]*)+\S*)”(?<代码>[^]*)(?[^]*)(?:“(?[^\”])”(?没有真正的帮助

它总是生成配置,而不给出测试结果


谢谢。

如果您有多个IP,您可以使用以下正则表达式:

^(?<host>[^ ]*(?:,\s+[^ ]+)*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$
              ^^^^^^^^^^^^^^


(?:\d+\){3}\d+(?:,\s*(?:\d+\){3}\d+)*
匹配
编号
+
+
编号
+
+
编号
+
编号
+
编号
,可选的相同模式用逗号列出。

谢谢你这么多Stribizev,我到了这个网站,但没有成功地进行准确的测试。我现在可以测试所有情况,我现在就试试看了让我的日志配置工作,并让你知道。你建议的更安全的模式工作,这是我确定的最好方式,ip地址是好的/约定的。但是我如何调整第二个正则表达式以匹配缺少主机的情况?例如匹配此。----[13/Aug/2015:09:50:45+0000]“GET HTTP/1.1”4010“-“Java/1.8.0\u 45”我可以用第一个正则表达式来做,但我用第二个正则表达式做了尝试(对我来说是正确的),但没有成功。ThanksI发现,这是如何做的:^(?)((\D;)((?:\D+\){3})\D+(?:,\s*(?:\D+\){3}\D+*)[^]*(?[^]*)[((?\s+((?+)((?:+)((((?:+)+)(((((?)+)((((?)(((?)+)+)(((((((?)+)+)(((((?)+)+)+)((((?)((((?)+)+)+)+)+)((((?)(((((((?)+)+))+)+)?$Thanking on SO在左侧表示感谢,如果答案对您有效。是的,我是在发表评论后才这样做的。我还阅读了StackExchange文档以了解这一点。我完全同意,并且知道下次如何做。谢谢。
 <source>
      type tail
      format format /^(?<host>\,*[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$/ 
      time_format %d/%b/%Y:%H:%M:%S %z
      path /var/log/varnish/varnishncsa.log
      pos_file /var/log/td-agent/tmp/access.log.pos
      tag "apache2.varnish.mydomain.com.access"
    </source>
^(?<host>[^ ]*(?:,\s+[^ ]+)*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$
              ^^^^^^^^^^^^^^
^(?<host>(?:\d+\.){3}\d+(?:,\s*(?:\d+\.){3}\d+)*|-) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$