Elkstack Logstash-如何通过电子邮件发送阈值警报

我需要一些关于Logstash的帮助。我目前有下面的Logstash配置，它可以工作。当[message]标记中有“Token validation failed”（令牌验证失败）时，它会发送一封电子邮件，说明身份验证问题

 input {

  tcp {
    codec => "json"
    port => 5144
    tags => ["windows","nxlog"]
    type => "nxlog-json"
  }

} # end input

filter {

  if [type] == "nxlog-json" {
    date {
      match => ["[EventTime]", "YYYY-MM-dd HH:mm:ss"]
      timezone => "Europe/London"
    }
    mutate {
        rename => [ "AccountName", "user" ]
        rename => [ "AccountType", "[eventlog][account_type]" ]
        rename => [ "ActivityId", "[eventlog][activity_id]" ]
        rename => [ "Address", "ip6" ]
        rename => [ "ApplicationPath", "[eventlog][application_path]" ]
        rename => [ "AuthenticationPackageName", "[eventlog][authentication_package_name]" ]
        rename => [ "Category", "[eventlog][category]" ]
        rename => [ "Channel", "[eventlog][channel]" ]
        rename => [ "Domain", "domain" ]
        rename => [ "EventID", "[eventlog][event_id]" ]
        rename => [ "EventType", "[eventlog][event_type]" ]
        rename => [ "File", "[eventlog][file_path]" ]
        rename => [ "Guid", "[eventlog][guid]" ]
        rename => [ "Hostname", "hostname" ]
        rename => [ "Interface", "[eventlog][interface]" ]
        rename => [ "InterfaceGuid", "[eventlog][interface_guid]" ]
        rename => [ "InterfaceName", "[eventlog][interface_name]" ]
        rename => [ "IpAddress", "ip" ]
        rename => [ "IpPort", "port" ]
        rename => [ "Key", "[eventlog][key]" ]
        rename => [ "LogonGuid", "[eventlog][logon_guid]" ]
        rename => [ "Message", "message" ]
        rename => [ "ModifyingUser", "[eventlog][modifying_user]" ]
        rename => [ "NewProfile", "[eventlog][new_profile]" ]
        rename => [ "OldProfile", "[eventlog][old_profile]" ]
        rename => [ "Port", "port" ]
        rename => [ "PrivilegeList", "[eventlog][privilege_list]" ]
        rename => [ "ProcessID", "pid" ]
        rename => [ "ProcessName", "[eventlog][process_name]" ]
        rename => [ "ProviderGuid", "[eventlog][provider_guid]" ]
        rename => [ "ReasonCode", "[eventlog][reason_code]" ]
        rename => [ "RecordNumber", "[eventlog][record_number]" ]
        rename => [ "ScenarioId", "[eventlog][scenario_id]" ]
        rename => [ "Severity", "level" ]
        rename => [ "SeverityValue", "[eventlog][severity_code]" ]
        rename => [ "SourceModuleName", "nxlog_input" ]
        rename => [ "SourceName", "[eventlog][program]" ]
        rename => [ "SubjectDomainName", "[eventlog][subject_domain_name]" ]
        rename => [ "SubjectLogonId", "[eventlog][subject_logonid]" ]
        rename => [ "SubjectUserName", "[eventlog][subject_user_name]" ]
        rename => [ "SubjectUserSid", "[eventlog][subject_user_sid]" ]
        rename => [ "System", "[eventlog][system]" ]
        rename => [ "TargetDomainName", "[eventlog][target_domain_name]" ]
        rename => [ "TargetLogonId", "[eventlog][target_logonid]" ]
        rename => [ "TargetUserName", "[eventlog][target_user_name]" ]
        rename => [ "TargetUserSid", "[eventlog][target_user_sid]" ]
        rename => [ "ThreadID", "thread" ]

    }
    mutate {
        remove_field => [
                    "CurrentOrNextState",
                    "Description",
                    "EventReceivedTime",
                    "EventTime",
                    "EventTimeWritten",
                    "IPVersion",
                    "KeyLength",
                    "Keywords",
                    "LmPackageName",
                    "LogonProcessName",
                    "LogonType",
                    "Name",
                    "Opcode",
                    "OpcodeValue",
                    "PolicyProcessingMode",
                    "Protocol",
                    "ProtocolType",
                    "SourceModuleType",
                    "State",
                    "Task",
                    "TransmittedServices",
                    "Type",
                    "UserID",
                    "Version"
                    ]
    }
  }

}

output {
   elasticsearch {
    hosts => ["localhost:9200"]
    }

if "Token validation failed" in [message]  {

email {

address => "smtp01.domain.com"
to => "example@domain.com"
from => "Sender@domain.com"
subject => "Auth Issue"
body => "Auth Issue"   
port => 25
use_tls => false
via => "smtp" 

}   
}

} # end output

---

我想知道，只有在一分钟内消息标签“令牌验证失败”10次时，如何让电子邮件发送。如果它有9个或以下条目，它将不会发送任何电子邮件。我需要设置什么配置才能使其工作？

有几种方法可以实现这一点

A.您可以使用XPack Alerting（以前称为Watcher）或ElastAlert，如中所述

B.您可以使用以跟踪和统计“令牌验证失败”消息，如中所述。你只需要

  aggregate {
    task_id => "%{[eventlog][target_logonid]}"
    code => "map['failed_count'] ||= 0; map['failed_count'] += 1;"
    push_map_as_event_on_timeout => true
    timeout => 60 # 1 minute timeout
    timeout_tags => ['_aggregatetimeout']
    timeout_code => "event.set('token_failed', event.get('failed_count') >= 10)"
  }

如果[token_失败]

C.您可以使用来计算和缓存“令牌验证失败”消息发生的次数。它基本上与B相同，但是通过在Ruby代码中自己实现逻辑

D.您可以使用来计算消息字段中“令牌验证失败”的事件发生率

  metrics {
    meter => [ "message" ]
    rates => [ 1 ]
    add_tag => "metric"
  }

然后在输出中，您可以简单地使用计量信息，如下所示：

  if "metric" in [tags] and [Token validation failed][count] >= 10 {
    email {
      ...
    }
  }

请注意，对于解决方案B和C，您无法使用启动日志存储（即

-w1

）。我已经提交了一份文件来“修复”这个问题，但由于Logstash团队已经有了大量的待办事项，我们将拭目以待