Logstash 使用logstatsh解析json的日志文件

我在日志文件中有如下json对象日志

{"con":"us","sl":[[1,2]],"respstats_1":"t:2,ts:140,m:192.168.7.5,p:|mobfox:1,P,E,0,0.4025:0.0:-:-,0-98;appnexus-marimedia:2,P,L,140,0.038:0.0:-:-,-;","rid":"AKRXRWLYCZIDFM","stats":"t:2,h:2,ts:140,mobfox:0,appnexus-marimedia:140,;m:192.168.7.5;p:","resp_count":0,"client_id":"15397682","err_stats":"mobfox:0-98,"}
{"con":"br","sl":[[1,2,3,4]],"respstats_1":"t:4,ts:285,m:192.168.7.5,p:|smaato:1,P,M,143,0.079:0.0:-:-,-;vserv-specialbuy:2,P,W,285,0.0028:0.0:-:-,-;mobfox:3,P,E,42,0.077:0.0:-:-,0-98;inmobi-pre7:4,P,H,100,0.0796:0.0:-:-,-;","rid":"AKRXRWLYCY4DOU","stats":"t:4,h:4,ts:285,smaato:143,vserv-specialbuy:285,mobfox:42,inmobi-pre7:100,;m:192.168.7.5;p:","resp_count":1,"client_id":"15397682","err_stats":"mobfox:0-98,","ads":[{"pricing":{"price":"0","type":"cpc"},"rank":2,"resp_json":{"img_url":"http://img.vserv.mobi/i/320x50_7/7bfffd967a91e0e38ee06ffcee1a75e5.jpg?108236_283989_c46e3f74","cli_url":"http://c.vserv.mobi/delivery/ck.php?p=2__b=283989__zoneid=108236__OXLCA=1__cb=c46e3f74__dc=1800__cd=usw3_uswest2a-1416567600__c=37742__rs=0a587520_15397682__mser=cdn__dat=3__dacp=12__zt=s__r=http%3A%2F%2Fyeahmobi.go2cloud.org%2Faff_c%3Foffer_id%3D28007%26aff_id%3D10070%26aff_sub%3D108236_283989_c46e3f74","beacons":["http://img.vserv.mobi/b.gif"],"ad_type":"image"},"resp_code":200,"resp_html":"<a href=\"http://c.vserv.mobi/delivery/ck.php?p=2__b=283989__zoneid=108236__OXLCA=1__cb=c46e3f74__dc=1800__cd=usw3_uswest2a-1416567600__c=37742__rs=0a587520_15397682__mser=cdn__dat=3__dacp=12__zt=s__r=http%3A%2F%2Fyeahmobi.go2cloud.org%2Faff_c%3Foffer_id%3D28007%26aff_id%3D10070%26aff_sub%3D108236_283989_c46e3f74\"><img src=\"http://img.vserv.mobi/i/320x50_7/7bfffd967a91e0e38ee06ffcee1a75e5.jpg?108236_283989_c46e3f74\"  alt=\"\" /> <\/a><img src=\"http://img.vserv.mobi/b.gif\"  alt=\"\" />","tid":"vserv-specialbuy","bid":"576111"}]}

启动时，我无法看到任何类型的输出或错误

编辑：

我使用了以下生成输出的配置

input {
    file {
        codec => "json"
        type => "json"
        path => "/home/pp38/fetcher.log"
        sincedb_path => "/home/pp38/tmp/logstash/sincedb"
    }
}

filter {
  json {
    source => "message"
    target => "message"
  }
}


output {
  stdout { codec => rubydebug }
  elasticsearch {
        hosts => ["localhost:9200"]
    }

}

但我得到的是每个字段都被elasticsearch索引的输出

---

如何将整个json消息作为message:jsonContent追加到新字段中？

您可以使用简单的多行处理此问题，但对于您的情况，有一个更好的编解码器插件，名为

将输入一个包含多个json（每行一个）的源代码，并立即处理每个json

此编解码器将解码新行分隔的流式JSON。编码将发出一个以\n注释结尾的JSON字符串：如果源输入是面向行的JSON，例如redis或文件输入，请不要使用此编解码器。而是使用json编解码器。更多信息：此编解码器希望接收以换行符结尾的行的流（字符串）。文件输入将生成不带换行符的行字符串。因此，此编解码器无法与面向行的输入一起工作

---

如何将整个json消息作为

message:jsonContent

追加到新字段中？

input {
    file {
        codec => "json"
        type => "json"
        path => "/home/pp38/fetcher.log"
        sincedb_path => "/home/pp38/tmp/logstash/sincedb"
    }
}

filter {
  json {
    source => "message"
    target => "message"
  }
}


output {
  stdout { codec => rubydebug }
  elasticsearch {
        hosts => ["localhost:9200"]
    }

}