elasticsearch 在LogStash中写入@timestamp
我需要将UNIX时间戳字段的值写入elasticsearch 在LogStash中写入@timestamp,elasticsearch,logstash,logstash-configuration,logstash-file,elasticsearch,Logstash,Logstash Configuration,Logstash File,我需要将UNIX时间戳字段的值写入@timestamp,这样我就可以正确地索引流经logstash的数据,我已经让这个部分工作了。但是,我还要求@timestamp的值应该是插入时间。为此,我创建了一个临时字段,用于保存@timestamp的原始值 以下是我的工作内容: filter { csv { separator => " " # <- this white space is actually a tab, don't change it, it's
@timestamp
,这样我就可以正确地索引流经logstash的数据,我已经让这个部分工作了。但是,我还要求@timestamp
的值应该是插入时间。为此,我创建了一个临时字段,用于保存@timestamp
的原始值
以下是我的工作内容:
filter {
csv {
separator => " " # <- this white space is actually a tab, don't change it, it's already perfect
skip_empty_columns => true
columns => ["timestamp", ...]
}
# works just fine
mutate {
add_field => {
"tmp" => "%{@timestamp}"
}
}
# works just fine
date {
match => ["timestamp", "UNIX"]
target => "@timestamp"
}
# this works too
mutate {
add_field => {
"[@metadata][indexDate]" => "%{+YYYY-MM-dd}"
}
}
# @timestamp is not being set back to its original value
date {
match => ["tmp", "UNIX"]
target => "@timestamp"
}
# works just fine
mutate {
remove_field => ["tmp"]
}
}
output {
elasticsearch {
hosts => "elasticsearch:9200"
# this works
index => "indexname-%{[@metadata][indexDate]}"
}
}
@timestamp
未设置回其原始值。当我检查数据时,它的值与时间戳
字段的值相同。当您将日期添加到tmp
时,它将以ISO8601格式添加,因此您需要使用:
date {
match => ["tmp", "ISO8601"]
target => "@timestamp"
}
将日期添加到
tmp
时,它将以ISO8601格式添加,因此您需要使用:
date {
match => ["tmp", "ISO8601"]
target => "@timestamp"
}
你能简单地删除
target=>…
吗通常默认情况下会选择@timestamp
字段。我只是尝试了一下,先是在问题区域,然后在两个地方@timestamp
仍保留timestamp
的值。您可以简单地删除目标=>…
通常默认情况下会选择@timestamp
字段。我只是尝试了一下,首先在问题区域,然后在两个位置@timestamp
仍保留timestamp
的值。