Fluentd apache格式[警告]:模式不匹配:

Fluentd apache格式[警告]:模式不匹配:,apache,fluentd,Apache,Fluentd,在my/etc/fluent/fluent.conf中 <source> @type tail format apache2 path /var/log/apache2/other_vhosts_access.log tag apache2.access </source> @型尾 格式apache2 路径/var/log/apache2/other_vhosts_access.log tagapache2.access 错误/警告:2016-02-

在my/etc/fluent/fluent.conf中

<source>
  @type tail
  format apache2
  path /var/log/apache2/other_vhosts_access.log
  tag apache2.access
</source>

@型尾
格式apache2
路径/var/log/apache2/other_vhosts_access.log
tagapache2.access
错误/警告:2016-02-11 00:59:10+0100[警告]:模式不匹配: “mybebsite.dz:443105.101.114.234---[11/Feb/2016:00:59:10+0100] \“POST/\uu es/\u all/\u搜索HTTP/1.1\”200 794 \“\”\“Mozilla/5.0(WindowsNT6.1;WOW64; rv:43.0)Gecko/20100101 Firefox/43.0“

为什么这个父亲不匹配?
最好。

看来tail插件不支持apache日志格式“vhost_combined”的格式,而是“combined”
按照以下方式更改apache配置文件如何:

 <source>
    @type tail
    format **/^([^ ]*) (?<host>[^ ]*) [^ ]* "(?<method>\S+) (?<path>[^ ]* +\S*)? (?<code>[^ ]*) (?<size>[^ ]*) (?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?/**
    path /var/log/apache2/access.log
    pos_file /var/log/td-agent/httpd.access.pos
    tag s3.apache.access
  </source>
  <filter **>
    @type grep
    <regexp>
      key path
      pattern \/aws\/project_launch\/view\/[\w\W]*
    </regexp>
  </filter>
  <match **>
    @type s3
    aws_key_id xxxxxx
    aws_sec_key xxxxxx
    s3_bucket bucketalifluentd
    s3_region eu-west-1
    path logs_viewshare/
    buffer_path /var/log/td-agent/buffer/s3
    time_slice_format %Y-%m-%d/%H
    time_slice_wait 2m
  </match>
/etc/apache2/conf可用/其他vhosts访问日志。conf
之前:
CustomLog${APACHE_LOG_DIR}/other_vhost_access.LOG vhost_combined

(将vhost_combined更改为combined

在:
CustomLog${APACHE_LOG_DIR}/other_vhosts_access.LOG合并后

我在fluentd中遇到了警告:模式不匹配,因此我的筛选器部分无法工作。然后我认真对待这个警告,通过创建一个正则表达式解决了这个问题。因此,我的td-agent.conf如下所示:

 <source>
    @type tail
    format **/^([^ ]*) (?<host>[^ ]*) [^ ]* "(?<method>\S+) (?<path>[^ ]* +\S*)? (?<code>[^ ]*) (?<size>[^ ]*) (?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?/**
    path /var/log/apache2/access.log
    pos_file /var/log/td-agent/httpd.access.pos
    tag s3.apache.access
  </source>
  <filter **>
    @type grep
    <regexp>
      key path
      pattern \/aws\/project_launch\/view\/[\w\W]*
    </regexp>
  </filter>
  <match **>
    @type s3
    aws_key_id xxxxxx
    aws_sec_key xxxxxx
    s3_bucket bucketalifluentd
    s3_region eu-west-1
    path logs_viewshare/
    buffer_path /var/log/td-agent/buffer/s3
    time_slice_format %Y-%m-%d/%H
    time_slice_wait 2m
  </match>

@型尾
格式**/^([^]*)(?[^]*)[^]*“(?\S+([^]*+\S*)?(?
[^]*)(?[^]*)(?:”(?[^\“]*)”(?)

用于在ruby中创建正则表达式,因为fluentd/td代理是用ruby编写的。
之后,您可以查看在目录/var/log/td-agent/buffer/s3.xxx

中收集日志的缓冲区,我已经使用了rubular并将结果(工作)表达式粘贴到了td-agent.conf中。但是,在Windows上运行fluentd并读取Linux Docker转发的stdout仍然无法解析。