elasticsearch Logstash jdbc输入插件在将数据索引到postgres的弹性搜索时给出日期映射错误,elasticsearch,logstash,logstash-jdbc,elasticsearch,Logstash,Logstash Jdbc" /> elasticsearch Logstash jdbc输入插件在将数据索引到postgres的弹性搜索时给出日期映射错误,elasticsearch,logstash,logstash-jdbc,elasticsearch,Logstash,Logstash Jdbc" />

elasticsearch Logstash jdbc输入插件在将数据索引到postgres的弹性搜索时给出日期映射错误

elasticsearch Logstash jdbc输入插件在将数据索引到postgres的弹性搜索时给出日期映射错误,elasticsearch,logstash,logstash-jdbc,elasticsearch,Logstash,Logstash Jdbc,背景: 我正在尝试将Postgres中的数据索引到弹性搜索中,并执行以下步骤: 步骤1:-使用插件从Postgres批量导入数据。 步骤2:-使用弹性搜索REST API(针对CRUD)从应用程序同步进一步的元数据更改 开始时,我在弹性搜索中创建了文档映射,如下所示:- {"metatestsample": { "properties": { "business_number": { "type": "long"

背景: 我正在尝试将Postgres中的数据索引到弹性搜索中,并执行以下步骤:

步骤1:-使用插件从Postgres批量导入数据。 步骤2:-使用弹性搜索REST API(针对CRUD)从应用程序同步进一步的元数据更改

开始时,我在弹性搜索中创建了文档映射,如下所示:-

   {"metatestsample": {
        "properties": {
           "business_number": {
              "type": "long"
           },
           "business_number_type": {
              "type": "string",
              "index":"not_analyzed"
           },
           "document_id": {
              "type": "long"
           },
           "document_location": {
              "type": "string",
              "index":"not_analyzed"
           },
           "document_number": {
              "type": "string",
              "index":"not_analyzed"
           },
           "document_status": {
              "type": "string",
              "index":  "not_analyzed"
           },
           "country": {
              "type": "string",
              "index":  "not_analyzed"
           },
           "document_created": {
              "type": "date",
              "format": "yyyy-MM-dd'T'HH:mm:ss"
           },
           "customer": {
              "properties": {
                 "customer_id": {
                    "type": "long"
                 },
                 "customer_number": {
                    "type": "string",
                    "index":  "not_analyzed"
                 },
                 "customer_name": {
                    "type": "string",
                    "index":  "not_analyzed"
                 },
                 "address1":{
                    "type": "string",
                    "index":  "not_analyzed"
                 },
                 "address2":{
                    "type": "string",
                    "index":  "not_analyzed"
                 },
                 "city":{
                    "type": "string",
                    "index":  "not_analyzed"
                 },
                 "state":{
                    "type": "string",
                    "index":  "not_analyzed"
                 },
                 "zip":{
                    "type": "string",
                    "index":  "not_analyzed"
                 },
                 "country":{
                    "type": "string",
                    "index":  "not_analyzed"
                 },
                 "phone":{
                    "type": "string",
                    "index":  "not_analyzed"
                 },
                 "fax":{
                    "type": "string",
                    "index":  "not_analyzed"
                 },
                 "email":{
                    "type": "string",
                    "index":  "not_analyzed"
                 },
                 "contact_name":{
                    "type": "string",
                    "index":  "not_analyzed"
                 },
                 "customer_created":{
                    "type": "date",
                    "format": "yyyy-MM-dd'T'HH:mm:ss"
                 },
                 "customer_modified":{
                    "type": "date",
                    "format": "yyyy-MM-dd'T'HH:mm:ss"
                 },
                 "type":{
                    "type": "string",
                    "index":  "not_analyzed"
                 }
              }
           },

           "expiration_date": {
              "type": "date",
              "format": "YYYY-MM-DD"
           },
           "legacy_document_id": {
              "type": "string"
           },
           "document_modified": {
              "type": "date",
              "format": "yyyy-MM-dd'T'HH:mm:ss"
           },
           "review_date": {
              "type": "date",
              "format": "YYYY-MM-DD"
           },
           "valid": {
              "type": "boolean"
           },
           "invalid_reason": {
              "type": "string",
              "index":  "not_analyzed"
           }
        }
     } }
并为jdbc输入插件添加了logstash配置:-

  input{
    jdbc {
         # Postgres jdbc connection string to our database, mydb
         jdbc_connection_string => "jdbc:postgresql://localhost:5432/mydb"
         # The user we wish to execute our statement as
         jdbc_user => "user"
         jdbc_password => "password"
         # The path to our downloaded jdbc driver
         jdbc_driver_library => "/data/logstash/postgresql-9.4-1204.jdbc4.jar"
         # The name of the driver class for Postgresql
         jdbc_driver_class => "org.postgresql.Driver"
         jdbc_validate_connection => true
         # our query
         **statement_filepath => "testindex.sql"**
     }
  }

 output{
   elasticsearch{
     action =>"update"
     index => "indexname"
     manage_template => false
     document_id => "%{uid}"
     doc_as_upsert => true
     hosts => ["192.168.56.105"]
   }
 }
注: 语句\u filepath=>“testindex.sql” testindex.sql具有select语句,用于从postgres查询数据

问题: 当我尝试使用RESTAPI将数据直接插入弹性搜索时,它可以正常工作,并且文档插入时不会出现任何错误

**但当我尝试使用上面的LogstashJDBC插件插入文档时,它给出了下面的错误。我是否缺少任何标志或配置参数?**

status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Mapper for [expiration_date] conflicts with existing mapping in other types:\n[mapper [expiration_date] is used by multiple types. Set update_all_types to true to update [format] across all types.]"}}}, :level=>:warn}

您真的在ES中安装了名为
metatestsample
的索引模板吗?请共享以下命令的输出:
curl-XGET localhost:9200/\u template
output is{}谢谢,我意识到template\u name参数是不需要的,我用参数manage\u template=>false更新了配置,但仍然得到相同的错误您可能需要删除索引并正确地重新创建它,您似乎有冲突的映射类型。唯一的出路是一个干净的索引/映射。谢谢,它工作得很好。