使用logstash将json文件数据导入弹性搜索

使用logstash将json文件数据导入弹性搜索,logstash,logstash-grok,logstash-configuration,Logstash,Logstash Grok,Logstash Configuration,我想将json文件数据导入弹性搜索。 这是我的logstash配置文件- 输入{文件{ 类型=>json path=>C:\Users\Desktop\newJSON.json 开始位置=>开始sincedb\u路径=>\dev\null }} 输出{ stdout{ 编解码器=>rubydebug } 弹性搜索{ hosts=>localhost:9200 index=>jsondata1 }} 这是我的json文件-- { 水果:苹果, 尺寸:小, 颜色:红色 }, { 水果:木瓜, 尺寸:

我想将json文件数据导入弹性搜索。 这是我的logstash配置文件-

输入{文件{ 类型=>json path=>C:\Users\Desktop\newJSON.json 开始位置=>开始sincedb\u路径=>\dev\null }}

输出{ stdout{ 编解码器=>rubydebug } 弹性搜索{ hosts=>localhost:9200 index=>jsondata1 }}

这是我的json文件--

{ 水果:苹果, 尺寸:小, 颜色:红色 }, { 水果:木瓜, 尺寸:大, 颜色:黄色 测试:甜味 }

我使用这个命令执行了上面的配置文件--

logstash -f logstashcon.conf
但我在弹性搜索索引中得到了如下数据-

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 10,
    "max_score": 1,
    "hits": [
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablA",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.302Z",
          "@version": "1",
          "host": "user-102",
          "message": "{\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablB",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.694Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"fruit\": \"Apple\",\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablE",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.696Z",
          "@version": "1",
          "host": "user-102",
          "message": "},\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablC",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.695Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"size\": \"Large\",\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablD",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.696Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"color\": \"Red\"\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablG",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.698Z",
          "@version": "1",
          "host": "user-102",
          "message": "\"fruit\": \"Papaya\",\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablJ",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.699Z",
          "@version": "1",
          "host": "user-102",
          "message": "}\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablH",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.699Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"size\": \"Large\",\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablF",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.698Z",
          "@version": "1",
          "host": "user-102",
          "message": "{\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablI",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.699Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"color\": \"Yellow\"\r",
          "type": "json"
        }
      }
    ]
  }
}
请帮助我获得正确的输出
谢谢大家!

您需要使用logstash的JSON过滤器插件来构造输出

例如,如果消息字段中有JSON数据,则过滤器将为

filter {
  json {
    source => "message"
  }
}
文件上是这么说的

它接受包含JSON的现有字段并将其扩展为 Logstash事件中的实际数据结构


请阅读更多信息。

阅读日志存储手册。也要学会设置你的问题的格式,因为它几乎看不清