elasticsearch 日志存储配置如何将aws s3 csv无标题传输到Elasticsearch

我有s3中的示例csv文件，有3列，没有任何标题。但是在从S3CSV到elasticsearch的数据传输过程中，我想给每一列指定一些名称（在我的例子中，分别是0到2列的id、名称、年龄）

输入样本.csv

  1,myname,23
  2,myname2,24

预期输出应为ES索引中的以下文档：

        [{
            "_index": "user_detail",
            "_type": "user_detail_type",
            "_id": "1",
            "_score": 1.0,
            "_source": {
                "id": "1",
                "name": "myname",
                "age": "23"
            }
        },
        {
            "_index": "user_detail",
            "_type": "user_detail_type",
            "_id": "2",
            "_score": 1.0,
            "_source": {
                "id": "2",
                "name": "myname2",
                "age": "24"
            }
        }]

input {
      s3 {
         bucket => "users"
         region => "us-east-1"
         watch_for_new_files => false
         prefix => "user.csv"
        }
}

filter {
  // Need help here
}

output {
     elasticsearch {
          hosts => "localhost:9200"
          index => "user_detail"
          document_type => "user_detail_type"
          document_id => "%{id}"
     }
}

我编写的日志存储配置是：

        [{
            "_index": "user_detail",
            "_type": "user_detail_type",
            "_id": "1",
            "_score": 1.0,
            "_source": {
                "id": "1",
                "name": "myname",
                "age": "23"
            }
        },
        {
            "_index": "user_detail",
            "_type": "user_detail_type",
            "_id": "2",
            "_score": 1.0,
            "_source": {
                "id": "2",
                "name": "myname2",
                "age": "24"
            }
        }]

input {
      s3 {
         bucket => "users"
         region => "us-east-1"
         watch_for_new_files => false
         prefix => "user.csv"
        }
}

filter {
  // Need help here
}

output {
     elasticsearch {
          hosts => "localhost:9200"
          index => "user_detail"
          document_type => "user_detail_type"
          document_id => "%{id}"
     }
}

怀疑：

---

在Elasticsearch插入期间，我应该在filter部分中写些什么，或者对配置进行任何更改，以转换列[0]=>id，列[1]=>name，列[2]=>age。

在输入中使用csv编解码器。您可以使用

columns

选项设置列名。我的问题是当我没有csv标题时，请您再解释一下@不带标头的BaudsParsing csv是csv编解码器的一个受支持的用例，因为您可以在codec@baudsp感谢它与列的csv过滤器一起工作。我以前用错了。我用下面的配置添加了过滤器部分。筛选器{csv{columns=>[“id”，“name”，“age”]分隔符=>“，”}