elasticsearch ELK堆栈中的IIS用户名

我正试图通过IIS日志和Elasticsearch解决一个问题。发生的情况是，IIS日志中的用户名中有反斜杠（），而不是正斜杠（/）。当Elasticsearch返回名称时，它不再具有\我希望在输入时能够转义的\名称。因此，在Elasticsearch或Kibana中查看结果时，用户名没有\并且斜杠被视为正则表达式。例如，用户名abcd\bob将作为abcdob返回

我还认为，这个问题将是我在来自IIS的每个条目中添加_grokparsefailure标记的原因

有什么建议吗

我的NXLOG文件正在获取数据：

## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> #<Extension w3c> #map iis log fields to Field Types # Module xm_csv # Fields $date, $time, $website, $serverip, $method, $url, $querystring, $port, $username, $clientip, $version, $useragent, $referer, $status, $substatus, $sc_win32_status, $sc_bytes, $cs_bytes, $time_taken # FieldTypes string, string, string, string, string, string, string, string, string, string, string, string, string, integer, integer, integer, integer, integer, integer # # #Fields $date, $time, $website, $hostname, $serverip, $verb, $request, $querystring, $dstport, $user, $clientip, $httpversion, $useragent, $cookie, $referrer, $fqdn, $status, $substatus, $sc_win32_status, $sc_bytes, $cs_bytes, $time_taken # #FieldTypes string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, integer, integer, integer, integer, integer, integer # Delimiter ' ' # QuoteChar '"' # EscapeControl FALSE # UndefValue - #</Extension> <Extension w3c> #map iis log fields to Field Types Module xm_csv Fields $date, $time, $website, $hostname, $serverip, $verb, $request, $querystring, $dstport, $user, $clientip, $httpversion, $useragent, $cookie, $referrer, $fqdn, $status, $substatus, $sc_win32_status, $sc_bytes, $cs_bytes, $time_taken FieldTypes string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, integer, integer, integer, integer, integer Delimiter ' ' </Extension> # Nxlog internal logs <Input internal> Module im_internal Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json(); </Input> # Windows Event Log <Input eventlog> # Uncomment im_msvistalog for Windows Vista/2008 and later Module im_msvistalog # Uncomment im_mseventlog for Windows XP/2000/2003 # Module im_mseventlog Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json(); </Input> <Input iis-logs> Module im_file File 'C:\inetpub\logs\LogFiles\W3SVC1\u_ex*.log' ReadFromLast TRUE Exec if $raw_event =~ /^#/ drop(); \ else \ { \ w3c->parse_csv(); \ $EventTime = parsedate($date + " " + $time); \ to_json (); \ } </Input> <Output out> Module om_tcp Host logs.{domain removed}.com Port 3515 </Output> <Output iis-out> Module om_tcp Host logs.{domain removed}.com Port 3516 </Output> <Route 1> Path internal, eventlog => out </Route> <Route 2> Path iis-logs => iis-out </Route> ##请将根目录设置为安装nxlog的文件夹， ##否则它不会启动。 #定义根C:\Program Files\nxlog 定义根C:\Program Files（x86）\nxlog Moduledir%ROOT%\modules CacheDir%ROOT%\data pid文件%ROOT%\data\nxlog.pid SpoolDir%ROOT%\data 日志文件%ROOT%\data\nxlog.log 模块xm_json # #将iis日志字段映射到字段类型 #模块xm_csv #字段$date、$time、$website、$serverip、$method、$url、$querystring、$port、$username、$clientip、$version、$useragent、$referer、$status、$substatus、$sc\u win32\u status、$sc\u bytes、$cs\u bytes、$time\u #字段类型string，string，string，string，string，string，string，string，string，string，string，string，string，string，integer，integer，integer，integer，integer # ##字段$date、$time、$website、$hostname、$serverip、$verb、$request、$querystring、$dsport、$user、$clientip、$httpversion、$useragent、$cookie、$referer、$fqdn、$status、$substatus、$sc_win32_status、$sc_字节、$cs_字节、$time_ ##字段类型字符串、字符串、字符串、字符串、字符串、字符串、字符串、字符串、字符串、字符串、字符串、字符串、字符串、字符串、字符串、整数、整数、整数、整数、整数 #分隔符“” #引述 #逃逸控制错误 #未定义值- # #将iis日志字段映射到字段类型 模块xm_csv 字段$date、$time、$website、$hostname、$serverip、$verb、$request、$querystring、$dsport、$user、$clientip、$httpversion、$useragent、$cookie、$referer、$fqdn、$status、$substatus、$sc_win32_status、$sc_字节、$cs_字节、$time_ 字段类型字符串，字符串，字符串，字符串，字符串，字符串，字符串，字符串，字符串，字符串，字符串，字符串，字符串，字符串，字符串，字符串，字符串，字符串，字符串，整数，整数，整数，整数，整数，整数 分隔符“” #Nxlog内部日志 模块im_内部 Exec$EventReceivedTime=integer（$EventReceivedTime）/1000000；to_json（）； #Windows事件日志 #取消对Windows Vista/2008及更高版本的im_msvistalog的注释 模块im_msvistalog #取消注释Windows XP/2000/2003的im_mseventlog #模块im_mseventlog Exec$EventReceivedTime=integer（$EventReceivedTime）/1000000；to_json（）； 模块im_文件 文件'C:\inetpub\logs\LogFiles\W3SVC1\u\u ex*.log' ReadFromLast TRUE 如果$raw_event=~/^#/drop（），则执行\ 否则\ { \ w3c->parse_csv（）\ $EventTime=parsedate（$date++$time）\ to_json（）\ } 模块om_tcp 主机日志。{domain removed}.com 端口3515 模块om_tcp 主机日志。{domain removed}.com 端口3516 路径内部，事件日志=>out 路径iis日志=>iis输出 我的Logstash.conf文件：

input { tcp { port => 5000 type => "syslog" } tcp { type => "eventlog" port => 3515 codec => json_lines } tcp { type => "iislog" port => 3516 codec => json_lines } } filter { if [type] == "syslog" { grok { match => { "message" => "<%{POSINT:syslog_pri}>%{DATA:syslog_timestamp} %{DATA:syslog_program}\[%{NUMBER:syslog_pid}\]\: %{GREEDYDATA:syslog_message}" } add_field => [ "received_at", "%{@timestamp}" ] # add_field => [ "received_from", "%{@source_host}" ] } syslog_pri { } date { match => [ "syslog_timestamp", "yyyy:MM:dd-HH:mm:ss" ] } if "_grokparsefailure" not in [tags] { mutate { replace => [ "@message", "%{syslog_message}" ] } } mutate { remove => [ "syslog_message", "syslog_timestamp" ] } kv { source => "@message" } } if [type] == "eventlog" { # Incoming Windows Event logs from nxlog # The EventReceivedTime field must contain only digits, or it is an invalid message # if [EventReceivedTime] !~ /\d+/ { drop { } } # grep { # match => [ "EventReceivedTime", "\d+" ] # } mutate { # Lowercase some values that are always in uppercase lowercase => [ "EventType", "FileName", "Hostname", "Severity" ] } mutate { # Set source to what the message says rename => [ "Hostname", "@source_host" ] } date { # Convert timestamp from integer in UTC match => [ "EventReceivedTime", "UNIX" ] } mutate { # Rename some fields into something more useful rename => [ "Message", "@message" ] rename => [ "Severity", "eventlog_severity" ] rename => [ "SeverityValue", "eventlog_severity_code" ] rename => [ "Channel", "eventlog_channel" ] rename => [ "SourceName", "eventlog_program" ] rename => [ "SourceModuleName", "nxlog_input" ] rename => [ "Category", "eventlog_category" ] rename => [ "EventID", "eventlog_id" ] rename => [ "RecordNumber", "eventlog_record_number" ] rename => [ "ProcessID", "eventlog_pid" ] } mutate { # Remove redundant fields remove => [ "SourceModuleType", "EventTimeWritten", "EventTime", "EventReceivedTime", "EventType" ] } if [eventlog_id] == 4624 { mutate { add_tag => [ "ad-logon-success" ] } } if [eventlog_id] == 4634 { mutate { add_tag => [ "ad-logoff-success" ] } } if [eventlog_id] == 4771 or [eventlog_id] == 4625 or [eventlog_id] == 4769 { mutate { add_tag => [ "ad-logon-failure" ] } } if [eventlog_id] == 4723 { mutate { add_tag => [ "ad-password-change" ] } } if [eventlog_id] == 4724 { mutate { add_tag => [ "ad-password-reset" ] } } if "ad-logon-success" in [tags] { metrics { add_tag => [ "drop", "metric", "ad-logon-success" ] meter => "ad-logon-success-metric" } } if "ad-logon-failure" in [tags] { metrics { add_tag => [ "drop", "metric", "ad-logon-failure" ] meter => "ad-logon-failure-metric" } } } if [type] == "iislog" { grok { # match => ["message", "%{TIMESTAMP_ISO8601:log_timestamp} %{WORD:iisSite} %{IPORHOST:site} %{IP:hostip} %{WORD:method} %{URIPATH:page} %{NOTSPACE:querystring} %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clientip} %{NOTSPACE:httpversion} %{NOTSPACE:useragent} %{NOTSPACE:referer} %{NUMBER:status} %{NUMBER:sub-status} %{NUMBER:win32-status} %{NUMBER:bytes-received} %{NUMBER:bytes-sent} %{NUMBER:time-taken}"] match => ["message", "%{DATESTAMP:log_timestamp} %{WORD:sitename} %{HOSTNAME:computername} %{IP:hostip} %{URIPROTO:method} %{URIPATH:request} (?:%{NOTSPACE:queryparam}|-) %{NUMBER:port} (?:%{NOTSPACE:username}|-) %{IP:clientip} %{NOTSPACE:httpversion} %{NOTSPACE:user-agent} (?:%{NOTSPACE:cookie}|-) (?:%{NOTSPACE:referer}|-) (?:%{HOSTNAME:host}|-) %{NUMBER:status} %{NUMBER:sub-status} %{NUMBER:win32-status} %{NUMBER:bytes-received} %{NUMBER:bytes-sent} %{NUMBER:time-taken}"] } useragent { source => "useragent" } #geoip { # source => "clientip" #} } metrics { meter => "events" add_tag => [ "drop", "metric", "events-metric" ] } } output { if "drop" not in [tags] { elasticsearch { host => "127.0.0.1" cluster => "logs" } # stdout { codec => rubydebug } } } 输入{ tcp{ 端口=>5000 类型=>“系统日志” } tcp{ 类型=>“事件日志” 端口=>3515 编解码器=>json\u行 } tcp{ 类型=>“iSlog” 端口=>3516 编解码器=>json\u行 } } 滤器{ 如果[类型]=“系统日志”{ 格罗克{ match=>{“message”=>“%{DATA:syslog\u timestamp}%{DATA:syslog\u program}\[%{NUMBER:syslog\u pid}\]\:%{greedyddata:syslog\u message} add_field=>[“received_at”，“%{@timestamp}”] #add_field=>[“received_from”，“%{@source_host}”] } syslog_pri{} 日期{ 匹配=>[“系统日志\u时间戳”，“yyyy:MM:dd HH:MM:ss”] } 如果“\u grokparsefailure”不在[标签]中{ 变异{ replace=>[“@message”，“%{syslog_message}”] } } 变异{ remove=>[“syslog\u消息”，“syslog\u时间戳”] } 千伏{ source=>“@message” } } 如果[类型]=“事件日志”{ #从nxlog传入的Windows事件日志 #EventReceivedTime字段只能包含数字，否则是无效消息 #如果[EventReceivedTime]！~/\d+/{drop{} #格雷普{ #匹配=>[“EventReceivedTime”，“\d+”] # } 变异{ #将某些始终为大写的值小写 小写=>[“事件类型”、“文件名”、“主机名”、“严重性”] } 变异{ #将source设置为消息所说的内容 重命名=>[“主机名”，“@source\u主机”] } 日期{ #从UTC中的整数转换时间戳 匹配=>[“E <Output iis2-out> Module om_file File 'C:\logs\logtest.txt' </Output>

EscapeChar 0x00