elasticsearch 为什么logstash要合并表?,elasticsearch,logstash,logstash-configuration,logstash-jdbc,elasticsearch,Logstash,Logstash Configuration,Logstash Jdbc" /> elasticsearch 为什么logstash要合并表?,elasticsearch,logstash,logstash-configuration,logstash-jdbc,elasticsearch,Logstash,Logstash Configuration,Logstash Jdbc" />

elasticsearch 为什么logstash要合并表?

elasticsearch 为什么logstash要合并表?,elasticsearch,logstash,logstash-configuration,logstash-jdbc,elasticsearch,Logstash,Logstash Configuration,Logstash Jdbc,我有两个配置文件,分别指向数据库的不同表,但在elasticsearch中,我合并了两个表,因此,tablea和tableb项都将作为typea复制,tablea和tableb项都将作为typeb复制 但我将其配置为将tablea作为typea,将tableb作为typeb 这是我的配置文件A input { jdbc { jdbc_driver_library => "/usr/share/logstash/mysql-connector-java-5.1.42

我有两个配置文件,分别指向数据库的不同表,但在elasticsearch中,我合并了两个表,因此,tablea和tableb项都将作为typea复制,tablea和tableb项都将作为typeb复制

但我将其配置为将tablea作为typea,将tableb作为typeb

这是我的配置文件A

input {

    jdbc {
        jdbc_driver_library => "/usr/share/logstash/mysql-connector-java-5.1.42-bin.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_connection_string => "jdbc:mysql://db:3306/nombase"
        jdbc_user => "root"
        jdbc_password => "root"
        schedule => "* * * * *"
        statement => "SELECT * FROM `tablecandelete`"
    }

}


## Add your filters / logstash plugins configuration here


output {

    stdout { codec => json_lines }

    if [deleted] == 1 {
        elasticsearch {
            hosts => "elasticsearch:9200"
            index => "nombase"
            document_type => "tablecandelete"

            ## TO PREVENT DUPLICATE ITEMS
            document_id => "tablecandelete-%{id}"
            action => "delete"
        }
    } else {
        elasticsearch {
            hosts => "elasticsearch:9200"
            index => "nombase"
            document_type => "tablecandelete"

            ## TO PREVENT DUPLICATE ITEMS
            document_id => "tablecandelete-%{id}"
        }
    }

}
这是我的配置文件B

input {

    jdbc {
        jdbc_driver_library => "/usr/share/logstash/mysql-connector-java-5.1.42-bin.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_connection_string => "jdbc:mysql://db:3306/nombase"
        jdbc_user => "root"
        jdbc_password => "root"
        schedule => "* * * * *"
        statement => "SELECT * FROM `nomtable`"
    }

}


## Add your filters / logstash plugins configuration here


output {

    stdout { codec => json_lines }

    elasticsearch {
        hosts => "elasticsearch:9200"
        index => "nombase"
        document_type => "nomtable"

        ## TO PREVENT DUPLICATE ITEMS
        document_id => "nomtable-%{id}"
    }

}

有人有主意吗?感谢您的回复/帮助虽然您有两个不同的配置文件,但logstash不会将它们视为独立的配置文件--您可以将所有输入放在一个文件中,将所有输出放在另一个文件中

要区分这些内容,您必须在每个输入上添加
type=>something\u unique
,然后用

if [type] == 'something_unique' { 
  // config here 
}

像这样的<如果[type]=“nomtable”{elasticsearch{hosts=>“elasticsearch:9200”index=>“nombase”document_type=>“nomtable”###防止重复项文档_id=>“nomtable-%{id}}}