如何在一个Elasticsearch索引中索引两个mongodb集合
我有两个mongodb集合,如何在一个Elasticsearch索引中索引两个mongodb集合,mongodb,elasticsearch,logstash,elastic-stack,Mongodb,elasticsearch,Logstash,Elastic Stack,我有两个mongodb集合,事务和用户,这是事务示例 { "_id" : ObjectId("5cdd391e1e4b8f0cb8e17d0f"), "txId" : "6910dc01ff167d90e8fe249ce68a5149f82d099345473fab02916068570974bd", "sender" : ObjectId("5cdbe473ca52557874005809"), "receiver" : ObjectId("5cdd26a6b2d3700
事务
和用户
,这是事务示例
{ "_id" : ObjectId("5cdd391e1e4b8f0cb8e17d0f"), "txId" :
"6910dc01ff167d90e8fe249ce68a5149f82d099345473fab02916068570974bd",
"sender" : ObjectId("5cdbe473ca52557874005809"), "receiver" :
ObjectId("5cdd26a6b2d370061e8435d7"), "senderWalletId" :
ObjectId("5cdbe473ca5255787400580a"), "receiverWalletId" :
ObjectId("5cdd26a6b2d370061e8435d8"), "status" : "success", "type" :
"transfer", "amount" : 3000, "totalFee" : 40, "createdAt" :
ISODate("2019-05-16T10:19:10.809Z"), "updatedAt" :
ISODate("2019-05-16T10:19:10.809Z"), "__v" : 0 }
这是一个用户示例
{ "_id" : ObjectId("5d010a140f0c30757f59fe18"), "role" : "client",
"status" : "active", "isPhoneVerified" : true, "personalDocuments" : [
], "email" : "example.com", "firstName" : "mm", "lastName" : "mm",
"phoneNumber" : "0000000", "password" : "$2a$10$.", "pushToken" :
"cSRdsgJXAc67k-PIKwHvslINb0kAaStVzmYPqeIH5oudVTqppHFjxbGEg2B-
1Xe8P0iTH0EYB9PHbKKey", "created_at" :
ISODate("2019-06-12T14:20:04.338Z"), "updated_at" :
ISODate("2019-06-17T13:33:41.613Z"), "__v" : 0, "verifyPhoneCode" :
null }
我已经使用logstash和mongodb输入插件成功地为ES中的事务文档编制了索引,这是ES为此生成的映射
"puretransactionsmodified" : {
"mappings" : {
"doc" : {
"properties" : {
"@timestamp" : {
"type" : "date"
},
"@version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"__v" : {
"type" : "long"
},
"amount" : {
"type" : "long"
},
"createdAt" : {
"type" : "date"
},
"host" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"log_entry" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"logdate" : {
"type" : "date"
},
"mongo_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"receiver" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"receiverWalletId" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"sender" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"senderWalletId" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"status" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"total" : {
"type" : "long"
},
"totalFee" : {
"type" : "long"
},
"txId" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"updatedAt" : {
"type" : "date"
}
}
}
}
}
}
当我尝试使用同一个插件为两个集合编制索引进行分析时
input {
mongodb {
uri => 'mongodb://**.**.**.***:27017/db'
placeholder_db_dir => '/home/jhon/Desktop
/userstransactions'
placeholder_db_name => 'logstash_sqlite.db'
collection => 'transactions'
batch_size => 202
parse_method => "simple"
}
mongodb {
uri => 'mongodb://**.**.**.***:27017/db'
placeholder_db_dir => '/home/jhon/Desktop
/userstransactions'
placeholder_db_name => 'logstash_sqlite.db'
collection => 'wallets'
batch_size => 202
parse_method => "simple"
}
}
它不起作用,我所拥有的只是交易,我想在交易中的接收者
和发送者
字段下拥有用户
提供的接收者的名称
,我是这方面的新手,我自己也尝试过映射它,但没有成功