elasticsearch Logstash:查询elasticsearch时出错

各位好,

通过logstash，我想查询elasticsearch，以便从以前的事件中获取字段，并使用当前事件的字段进行一些计算，然后添加新字段。以下是我所做的：

输入文件：

{"device":"device1","count":5}  
{"device":"device2","count":11} 
{"device":"device1","count":8}  
{"device":"device3","count":100}
{"device":"device3","count":95} 
{"device":"device3","count":155}
{"device":"device2","count":15} 
{"device":"device1","count":55} 

{"device":"device1","count":5,"previousCount=0","delta":0}
{"device":"device2","count":11,"previousCount=0","delta":0}
{"device":"device1","count":8,"previousCount=5","delta":3}
{"device":"device3","count":100,"previousCount=0","delta":0}
{"device":"device3","count":95,"previousCount=100","delta":-5}
{"device":"device3","count":155,"previousCount=95","delta":60}
{"device":"device2","count":15,"previousCount=11","delta":4}
{"device":"device1","count":55,"previousCount=8","delta":47}

filter {
    elasticsearch {
        hosts => ["localhost:9200/device"]  
        query => 'device:"%{[device]}"'
        sort => "@timestamp:desc"
        fields => ['count','previousCount']
    } 
    if [previousCount]{
        ruby {
            code => "event[delta] = event[count] - event[previousCount]"
        }
    }
    else{
        mutate {
            add_field => { "previousCount" => "0" }
            add_field => { "delta" => "0" }
        }
    }
}

我的预期输出：

{"device":"device1","count":5}  
{"device":"device2","count":11} 
{"device":"device1","count":8}  
{"device":"device3","count":100}
{"device":"device3","count":95} 
{"device":"device3","count":155}
{"device":"device2","count":15} 
{"device":"device1","count":55} 

{"device":"device1","count":5,"previousCount=0","delta":0}
{"device":"device2","count":11,"previousCount=0","delta":0}
{"device":"device1","count":8,"previousCount=5","delta":3}
{"device":"device3","count":100,"previousCount=0","delta":0}
{"device":"device3","count":95,"previousCount=100","delta":-5}
{"device":"device3","count":155,"previousCount=95","delta":60}
{"device":"device2","count":15,"previousCount=11","delta":4}
{"device":"device1","count":55,"previousCount=8","delta":47}

filter {
    elasticsearch {
        hosts => ["localhost:9200/device"]  
        query => 'device:"%{[device]}"'
        sort => "@timestamp:desc"
        fields => ['count','previousCount']
    } 
    if [previousCount]{
        ruby {
            code => "event[delta] = event[count] - event[previousCount]"
        }
    }
    else{
        mutate {
            add_field => { "previousCount" => "0" }
            add_field => { "delta" => "0" }
        }
    }
}

日志存储过滤器部件：

{"device":"device1","count":5}  
{"device":"device2","count":11} 
{"device":"device1","count":8}  
{"device":"device3","count":100}
{"device":"device3","count":95} 
{"device":"device3","count":155}
{"device":"device2","count":15} 
{"device":"device1","count":55} 

{"device":"device1","count":5,"previousCount=0","delta":0}
{"device":"device2","count":11,"previousCount=0","delta":0}
{"device":"device1","count":8,"previousCount=5","delta":3}
{"device":"device3","count":100,"previousCount=0","delta":0}
{"device":"device3","count":95,"previousCount=100","delta":-5}
{"device":"device3","count":155,"previousCount=95","delta":60}
{"device":"device2","count":15,"previousCount=11","delta":4}
{"device":"device1","count":55,"previousCount=8","delta":47}

filter {
    elasticsearch {
        hosts => ["localhost:9200/device"]  
        query => 'device:"%{[device]}"'
        sort => "@timestamp:desc"
        fields => ['count','previousCount']
    } 
    if [previousCount]{
        ruby {
            code => "event[delta] = event[count] - event[previousCount]"
        }
    }
    else{
        mutate {
            add_field => { "previousCount" => "0" }
            add_field => { "delta" => "0" }
        }
    }
}

我的问题： 对于输入文件的每一行，我都会遇到以下错误：无法查询elasticsearch以查找以前的事件。。 似乎在logstash开始处理下一行之前，没有将每一行完全处理好的行放入elasticsearch

我不知道我的结论是否正确，如果正确，为什么会这样

那么，你知道我怎样才能解决这个问题吗

感谢您的关注和帮助

---

S

这个答案应该会有帮助：我在问：S！之前遵循了这个答案！我只是想完成描述，对于nil:NilClassGood，错误是：error=>NoMethodError:undefined方法'start\u with'。您的群集中是否有其他索引？如果是，您应该尝试我的分叉版本，以便能够仅指定需要查询的索引。我还有其他索引。我刚刚用了你的分叉版本。但我仍然有相同的错误：无法查询elasticsearch以查找以前的事件…：错误=>#哦，我看到了问题，字段没有正确指定（文档错误）：用这个

fields=>['count'，'previousCount']

替换

fields=>{count=>'count'previousCount=>'previousCount'}

这个答案应该会有帮助：我在询问之前遵循了这个答案：s！我只是想完成描述，对于nil:NilClassGood，错误是：error=>NoMethodError:undefined方法'start\u with'。您的群集中是否有其他索引？如果是，您应该尝试我的分叉版本，以便能够仅指定需要查询的索引。我还有其他索引。我刚刚用了你的分叉版本。但我仍然有相同的错误：无法查询elasticsearch以查找以前的事件…：错误=>#哦，我看到问题了，字段没有正确指定（文档错误）：用这个

字段=>['count'，'previousCount']

替换

字段=>{count=>'count'previousCount=>'previousCount'}