Logstash-如何按[标签]过滤

不同网站的Logstash按标签过滤

问题：我在一台IIS服务器中有多个网站。。我想为发送到logstash的每个日志文件添加一个“标记”

这是我的日志存储转发器配置

每个日志文件代表一个不同的网站。。因此，我想为这些日志中的每一个添加标记，并能够根据这个特定的标记进行过滤

“logs\svr05\ex*”

这是我的日志存储的IIS配置

filter {
    if [type] == "iis" {
            if [message] =~ "^#" {
                            drop {}
            }
            grok {
                    break_on_match => false
                    match => [
                            "message", "%{TIMESTAMP_ISO8601:log_timestamp} %{IPORHOST:s-sitename} %{IPORHOST:s-ip} %{URIPROTO:cs-method} %{URIPATH:cs-uri-stem} (?:%{NOTSPACE:cs_query}|-) %{NUMBER:src_port} %{NOTSPACE:cs_username} %{IP:clientip} %{NOTSPACE:useragent} %{NUMBER:sc-substatus} %{NUMBER:sc_win32_status} %{NUMBER:sc-bytes} %{NUMBER:cs-bytes} %{NUMBER:timetaken}"
                    ]
            }

            date {

                    locale => "en"
                    match => [ "log_timestamp", "YYYY-MM-dd HH:mm:ss" ]
                    target => "@timestamp"
                    timezone => "Indian/Maldives"
            }
            useragent {
                    source=> "useragent"
                    prefix=> "browser"
            }
            geoip {
                    source => "clientip"
                    target => "geoip"
                    add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
                    add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
            }

            mutate {
                    add_field => [ "src_ip", "%{clientip}" ]
                    convert => [ "[geoip][coordinates]", "float" ]
                    replace => [ "@source_host", "%{clientip}" ]
                    replace => [ "@message", "%{message}" ]
                    rename => [ "cs_method", "method" ]
                    rename => [ "cs_stem", "request" ]
                    rename => [ "useragent", "agent" ]
                    rename => [ "cs_username", "username" ]
                    rename => [ "sc_status", "response" ]
                    rename => [ "timetaken", "time_request" ]
           }
    }
}
filter
  {
    if [type] == "iis" {
            mutate {
                    remove_field => [ "clientip", "host", "hostname", "logtime" ]
            }
    }
}

假设我想发送不同应用的日志

app1.egov.mv app2.egov.mv

如何为这些不同的IIS应用程序添加标记？并在发现模块中对其进行过滤，以便使用标签为特定网站制作图表：|

问候,

---

Ismail

您已经知道如何添加

类型

字段，因此只需使用相同的方法添加另一个包含主机名称的字段：

{
  ...,
  "files": [
    {
      "paths": [
         "logs\\svr08\\ex*",
         "logs\\svr05\\ex*",
         "logs\\svr04\\ex*",
         "logs\\svr03\\ex*"
      ],
      "fields": {
        "type": "iis",
        "virtualhost": "app1.egov.mv"
      },
      "dead time": "24h" 
    }
  ]
}

显然，如果不同的日志文件模式适用于不同的服务器，则必须拆分配置：

{
  ...,
  "files": [
    {
      "paths": [
         "logs\\svr08\\ex*"
      ],
      "fields": {
        "type": "iis",
        "virtualhost": "app1.egov.mv"
      },
      "dead time": "24h" 
    },
    {
      "paths": [
         "logs\\svr05\\ex*"
      ],
      "fields": {
        "type": "iis",
        "virtualhost": "app2.egov.mv"
      },
      "dead time": "24h" 
    },
    ...
  ]
}

---

另一个选项（我更喜欢）是让web服务器本身在每个日志条目中包含主机名。

您已经知道如何添加

类型

字段，因此只需使用相同的方法添加另一个包含主机名的字段：

{
  ...,
  "files": [
    {
      "paths": [
         "logs\\svr08\\ex*",
         "logs\\svr05\\ex*",
         "logs\\svr04\\ex*",
         "logs\\svr03\\ex*"
      ],
      "fields": {
        "type": "iis",
        "virtualhost": "app1.egov.mv"
      },
      "dead time": "24h" 
    }
  ]
}

显然，如果不同的日志文件模式适用于不同的服务器，则必须拆分配置：

{
  ...,
  "files": [
    {
      "paths": [
         "logs\\svr08\\ex*"
      ],
      "fields": {
        "type": "iis",
        "virtualhost": "app1.egov.mv"
      },
      "dead time": "24h" 
    },
    {
      "paths": [
         "logs\\svr05\\ex*"
      ],
      "fields": {
        "type": "iis",
        "virtualhost": "app2.egov.mv"
      },
      "dead time": "24h" 
    },
    ...
  ]
}

另一个选择（我更喜欢）是让web服务器本身在每个日志条目中包含主机名