elasticsearch 将MariaDB中的多个表同步到Elasticsearch 7.5,elasticsearch,mariadb,kibana,elk,elasticsearch,Mariadb,Kibana,Elk" /> elasticsearch 将MariaDB中的多个表同步到Elasticsearch 7.5,elasticsearch,mariadb,kibana,elk,elasticsearch,Mariadb,Kibana,Elk" />

elasticsearch 将MariaDB中的多个表同步到Elasticsearch 7.5

elasticsearch 将MariaDB中的多个表同步到Elasticsearch 7.5,elasticsearch,mariadb,kibana,elk,elasticsearch,Mariadb,Kibana,Elk,我试图使用JDBC插件将数据库同步到Elasticsearch,但不知何故,这种配置不适用于多个语句(多个表) 此配置对于单个语句运行良好,但对于多个查询不起作用。我错过了什么?我需要将此数据同步到Elasticsearch中的单个索引。通过创建唯一的文档id解决了此问题 input { jdbc { jdbc_driver_library => "" jdbc_driver_class => "Java::org.mariadb.jdbc.Driver"

我试图使用JDBC插件将数据库同步到Elasticsearch,但不知何故,这种配置不适用于多个语句(多个表)


此配置对于单个语句运行良好,但对于多个查询不起作用。我错过了什么?我需要将此数据同步到Elasticsearch中的单个索引。

通过创建唯一的文档id解决了此问题

input {
  jdbc {
    jdbc_driver_library => ""
    jdbc_driver_class => "Java::org.mariadb.jdbc.Driver"
    jdbc_connection_string => "jdbc:mariadb://localhost:3306/DB"
    jdbc_user => "username"
    jdbc_password => "password"
    jdbc_paging_enabled => true
    tracking_column => "column1"
    use_column_value => true
    type => "type1"
    tracking_column_type => "numeric"
    schedule => "*/5 * * * * *"
    statement => "SELECT *, UNIX_TIMESTAMP(date_column) AS unix_ts_in_secs FROM table1 WHERE (UNIX_TIMESTAMP(date_column) >= :sql_last_value) ORDER BY date_column ASC"
  }
  jdbc {
    jdbc_driver_library => ""
    jdbc_driver_class => "Java::org.mariadb.jdbc.Driver"
    jdbc_connection_string => "jdbc:mariadb://localhost:3306/DB"
    jdbc_user => "username"
    jdbc_password => "password"
    jdbc_paging_enabled => true
    tracking_column => "column1"
    use_column_value => true
    type => "type2"
    tracking_column_type => "numeric"
    schedule => "*/5 * * * * *"
    statement => "SELECT *, UNIX_TIMESTAMP(date_column) AS unix_ts_in_secs FROM table2 WHERE (UNIX_TIMESTAMP(date_column) >= :sql_last_value) ORDER BY date_column ASC"
  }
}
filter {
  mutate {
    copy => { "id" => "[@metadata][_id]"}
    remove_field => ["id", "@version", "unix_ts_in_secs"]
  }
}
 output {
  elasticsearch {
      hosts => ["localhost:9200"]
      index => "test"
      document_id => "%{[@metadata][_id]}"
  }
}