elasticsearch logstash geoip不适用于IPv4
我使用logstash[version 2.2]将系统日志索引到elasticsearch中,我还使用geoip获取源和目标地址,但在某些日志中,geoip似乎不起作用elasticsearch logstash geoip不适用于IPv4,elasticsearch,logstash,elasticsearch,Logstash,我使用logstash[version 2.2]将系统日志索引到elasticsearch中,我还使用geoip获取源和目标地址,但在某些日志中,geoip似乎不起作用 **config file:** input { tcp { type => syslog port => 8001 } udp { type => syslog
**config file:**
input {
tcp {
type => syslog
port => 8001
}
udp {
type => syslog
port => 8001
}
filter {
if [type] == "syslog" {
grok {
match => {
"message" => "\<%{NUMBER:number}\>%{timestamp:timestamp} %{WORD:logType}: %{NUMBER:ruleNumber},%{NUMBER:subRuleNumber}%{DATA}%{NUMBER:tracker},%{WORD:realinterface},%{WORD:reasonForTheLogEntry},%{WORD:actionTakenThatResultedInTheLogEntry},%{WORD:directionOfTheTraffic},%{NUMBER:IPversion},%{DATA:class},%{DATA:flowLabel},%{NUMBER:hopLimit},%{WORD:protocol},%{NUMBER:protocolID},%{NUMBER:length},%{IPV6:srcIP},%{IPV6:destIP},%{NUMBER:srcPort},%{NUMBER:destPort},%{NUMBER:dataLength}"
}
add_field => { "event" => "name" }
}
}
geoip {
source => "srcIP"
target => "geoSrc"
}
geoip {
source => "destIP"
target => "geoDest"
}
geoip {
source => "icmpDetinationIP"
target => "icmpDest"
}
}
output {
csv {
fields => "message"
path => "/data/streamed-logs/%{[host]}-%{+YYYY-MM-dd}.log"
}
stdout {
codec => "rubydebug"
}
elasticsearch {
hosts => "address"
}
}
**address having problem with geoIP:**
**配置文件:**
输入{
tcp{
类型=>syslog
端口=>8001
}
udp{
类型=>syslog
端口=>8001
}
滤器{
如果[类型]=“系统日志”{
格罗克{
匹配=>{
“消息”=>“\%%{timestamp:timestamp}%%{WORD:logType}:%%{NUMBER:ruleNumber}},%%{NUMBER:subRuleNumber}%%{DATA}{NUMBER:tracker},%%{WORD:realinterface},%%{WORD:reasonforthegentry},%%{WORD WORD actiontakenthattresultedthegentry},%%{WORD WORD NUMBER:directionOfTheTraffic},%%{NUMBER:IPversion},%%{DATA:class},%%数据:类},%%数据:flowLabel},%%推理机},%%协议:{WORD{NUMBER:protocolID}、%%{NUMBER:length}、%%{IPV6:srcIP}、%%{IPV6:destIP}、%%{NUMBER:srcPort}、%%{NUMBER:destPort}、%%{NUMBER:dataLength}”
}
添加_字段=>{“事件”=>“名称”}
}
}
geoip{
source=>“srcIP”
目标=>“geoSrc”
}
geoip{
source=>“destp”
目标=>“大地测量”
}
geoip{
source=>“ICMPdateationIP”
target=>“icmpDest”
}
}
输出{
csv{
字段=>“消息”
path=>“/data/streamed logs/%{[host]}-%{+YYYY-MM-dd}.log”
}
stdout{
编解码器=>“rubydebug”
}
弹性搜索{
主机=>“地址”
}
}
**存在geoIP问题的地址:**
我无法获取e80::c0d3:531b:f0cf:f546格式的地址的geoIP您需要使用
IPV6
grok模式,而不是IPV4
grok {
match => {
"message" => "...%{IPV6:srcIP},%{IPV6:destIP},%{IPV6:icmpDetinationIP}..."
^ ^ ^
| | |
here here and here
}
}
您需要使用
IPV6
grok模式,而不是IPV4
grok {
match => {
"message" => "...%{IPV6:srcIP},%{IPV6:destIP},%{IPV6:icmpDetinationIP}..."
^ ^ ^
| | |
here here and here
}
}
部分
%{NOTSPACE:tos}
太贪婪,捕获的字段太多。此外,示例日志行与grok模式不匹配。您可以在上尝试使用新grok更新您的问题吗?3月3日00:59:56筛选器日志:716777216,,100000105,igb1,匹配,块,in,6,0x00,0xF12F0255,UDP,17524,fe80::1c7f:2ca0:6385:f414,ff02::fb,535353524-新消息e format我已经对groke格式进行了EDID,并且从中添加了一条新消息。实际上,您的grok只有一个小错误:%{timestamp:timestamp}
应该读取%{SYSLOGTIMESTAMP:timestamp}
。修复这个问题,它就会工作。%{NOTSPACE:tos}
太贪婪,捕获的字段太多。此外,示例日志行与grok模式不匹配。您可以在上尝试使用新grok更新您的问题吗?3月3日00:59:56筛选器日志:716777216,,100000105,igb1,匹配,块,in,6,0x00,0xF12F0255,UDP,17524,fe80::1c7f:2ca0:6385:f414,ff02::fb,535353524-新消息e format我已经对groke格式进行了EDID,还添加了一条来自的新消息。实际上,您的grok只有一个小错误:%{timestamp:timestamp}
应该读取%{SYSLOGTIMESTAMP:timestamp}
。修复这个错误,它就会工作。