elasticsearch 卡夫卡的日志:无法解码avro,elasticsearch,logstash,apache-kafka,avro,elasticsearch,Logstash,Apache Kafka,Avro" /> elasticsearch 卡夫卡的日志:无法解码avro,elasticsearch,logstash,apache-kafka,avro,elasticsearch,Logstash,Apache Kafka,Avro" />

elasticsearch 卡夫卡的日志:无法解码avro

elasticsearch 卡夫卡的日志:无法解码avro,elasticsearch,logstash,apache-kafka,avro,elasticsearch,Logstash,Apache Kafka,Avro,我正在尝试使用Kafka队列中的序列化avro事件。kafka队列使用简单的java生成器填充。为清楚起见,我将分享以下三个部分: Avro模式文件 {"namespace": "example.avro", "type": "record", "name": "User", "fields": [ {"name": "name", "type": "string"}, {"name": "favorite_number", "type": ["int", "null

我正在尝试使用Kafka队列中的序列化avro事件。kafka队列使用简单的java生成器填充。为清楚起见,我将分享以下三个部分:

Avro模式文件

{"namespace": "example.avro",
 "type": "record",
 "name": "User",
 "fields": [
     {"name": "name", "type": "string"},
     {"name": "favorite_number",  "type": ["int", "null"]},
     {"name": "favorite_color", "type": ["string", "null"]}
 ]
}
input {
        kafka {
                zk_connect => "localhost:2181"
                topic_id => "MemoryTest"
                type => "standard_event"
                group_id => "butiline_dash_prod"
        reset_beginning => true
        auto_offset_reset => smallest
        codec => {
                avro => {
                    schema_uri => "/opt/ELK/logstash-1.5.4/bin/user.avsc"
                }
            }
        } 
}

output {
    stdout { 
     codec => rubydebug 
     }
}
Java生产者代码片段(User.class使用avro工具生成)

问题

管道在logstash水平出现故障。当一个新事件被推送到Kafka中时,我在logstash控制台上看到以下内容:

Alyssa�blue {:exception=>#<NoMethodError: undefined method `decode' for ["avro", {"schema_uri"=>"/opt/ELK/logstash-1.5.4/bin/user.avsc"}]:Array>, :backtrace=>["/opt/ELK/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-1.0.0/lib/logstash/inputs/kafka.rb:169:in `queue_event'", "/opt/ELK/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-1.0.0/lib/logstash/inputs/kafka.rb:139:in `run'", "/opt/ELK/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:177:in `inputworker'", "/opt/ELK/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:171:in `start_input'"], :level=>:error}

Alyssa�blue{:exception=>#最终发现了错误。而不是这个(如Logstash网站上所建议的-)

正确的语法是(如插件文档中所建议的):

我猜语法已经改变了

Alyssa�blue {:exception=>#<NoMethodError: undefined method `decode' for ["avro", {"schema_uri"=>"/opt/ELK/logstash-1.5.4/bin/user.avsc"}]:Array>, :backtrace=>["/opt/ELK/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-1.0.0/lib/logstash/inputs/kafka.rb:169:in `queue_event'", "/opt/ELK/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-1.0.0/lib/logstash/inputs/kafka.rb:139:in `run'", "/opt/ELK/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:177:in `inputworker'", "/opt/ELK/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:171:in `start_input'"], :level=>:error}
codec => {
    avro => {
        schema_uri => "/opt/ELK/logstash-1.5.4/bin/user.avsc"
    }
}
codec =>   avro {
        schema_uri => "/opt/ELK/logstash-1.5.4/bin/user.avsc"
}