Logstash 在不同的redis键中记录输出
我使用logstash从filebeat读取日志并写入redis。我来到了一个场景,我想过滤日志并存储在2个redis密钥中,其中一个密钥将存储日志的某些部分,另一个密钥将存储日志的某些其他部分。 这是我的日志格式Logstash 在不同的redis键中记录输出,logstash,logstash-grok,filebeat,elk,logstash-file,Logstash,Logstash Grok,Filebeat,Elk,Logstash File,我使用logstash从filebeat读取日志并写入redis。我来到了一个场景,我想过滤日志并存储在2个redis密钥中,其中一个密钥将存储日志的某些部分,另一个密钥将存储日志的某些其他部分。 这是我的日志格式 TRANSACTION::test_user::MSGID-aa0313ea-2d26-47b1-a2c7-207d969a1298::{"message":[{"recipientType":"individual",&q
TRANSACTION::test_user::MSGID-aa0313ea-2d26-47b1-a2c7-207d969a1298::{"message":[{"recipientType":"individual","recipient":"919876543210","xApiheader":"asdf","typeText":[],"typeTemplate":[{"name":"template_name","attributes":["testing","123"]}],"typeContact":[],"typeLocation":[]}],"username":"test_user","msgId":"aa0313ea-2d26-47b1-a2c7-207d969a1298"}::CLIENT_DISPLAY_ID-gBEGkZmHmWIxAgk8BPgSu2GrihM
这是我的日志代码
input {
beats {
port => 5044
}
}
filter
{
if "TRANSACTION::" in [message]
{
grok
{
match => [ "message" , "TRANSACTION::%{GREEDYDATA:username}::MSGID-%{GREEDYDATA:messageId}::%{GREEDYDATA:messageJson}::ERRORCODE-%{GREEDYDATA:errorCode}::ERRORMESSAGE-%{GREEDYDATA:errorMessage}" ]
}
grok
{
match => [ "message" , "TRANSACTION::%{GREEDYDATA:username}::MSGID-%{GREEDYDATA:messageId}::%{GREEDYDATA:messageJson}::CLIENT_DISPLAY_ID-%{GREEDYDATA:clientMessageId}" ]
}
mutate
{
add_field => { "CLIENT" => "%{username}" }
add_field => { "INDEX_TYPE" => "MESSAGE" }
remove_field => "beat"
remove_field => "tags"
remove_field => "host"
remove_field => "prospector"
remove_field => "input"
remove_field => "meta"
remove_field => "offset"
remove_field => "log"
}
}
}
output {
if [INDEX_TYPE] == "MESSAGE"
{
redis {
data_type => "list"
host => ["localhost:6379"]
key => "QUEUE_MESSAGE_TRANS"
}
}
#stdout { }
}
现在,这个脚本将消息正确地存储在redis列表“QUEUE\u message\u TRANS”中。但是现在我想将该日志中的xApiheader存储在另一个redis key in key=>值对中,例如:set xApiheader\u gbegkzmwixagk8bpgsu2grim asdf
其中gbegkzmwixagk8bpgsu2grim是日志中的客户端显示ID