Warning: file_get_contents(/data/phpspider/zhask/data//catemap/6/mongodb/12.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/hadoop/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
mongo hadoop。不处理mongodb文档删除_Mongodb_Hadoop_Hive_Mongodb Hadoop - Fatal编程技术网

mongo hadoop。不处理mongodb文档删除

mongo hadoop。不处理mongodb文档删除,mongodb,hadoop,hive,mongodb-hadoop,Mongodb,Hadoop,Hive,Mongodb Hadoop,我想同步mongodb和hadoop,但当我从mongodb中删除文档时,hadoop中不能删除此文档 我试着使用mongo hadoop和hive。这是配置单元查询: CREATE EXTERNAL TABLE SubComponentSubmission ( id STRING, status INT, providerId STRING, dateCreated TIMESTAMP, subComponentId STRING, packageName STRING

我想同步mongodb和hadoop,但当我从mongodb中删除文档时,hadoop中不能删除此文档

我试着使用mongo hadoop和hive。这是配置单元查询:

CREATE EXTERNAL TABLE SubComponentSubmission
(
  id STRING,
  status INT,
  providerId STRING,
  dateCreated TIMESTAMP,
  subComponentId STRING,
  packageName STRING
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
WITH SERDEPROPERTIES('mongo.columns.mapping'=
                    '{"id":"_id", "status":"Status", 
                      "providerId":"ProviderId", 
                      "dateCreated":"DateCreated", 
                      "subComponentId":"SubComponentPackage.SubComponentId", 
                      "packageName":"SubComponentPackage.PackageName"}'
                    )
TBLPROPERTIES('mongo.uri'='mongodb://<host>:27017/<db name>.<collection name>');
创建外部表子组件提交
(
id字符串,
状态INT,
providerId字符串,
dateCreated时间戳,
子组件TID字符串,
包装名称字符串
)
由'com.mongodb.hadoop.hive.mongostragehandler'存储
使用SerdeProperty('mongo.columns.mapping'=
“{”id:“_id”,“status:”status“,
“providerId”:“providerId”,
“dateCreated”:“dateCreated”,
“subComponentId”:“SubComponentPackage.subComponentId”,
“packageName”:“SubComponentPackage.packageName”}
)
TBLProperty('mongo.uri'='mongodb://:27017/);
此查询创建与相应mongodb集合同步的表。通过这个查询,mongo hadoop也可以处理文档删除

mongo hadoop是否有任何选项不处理文档删除? 或者,有没有其他工具可以解决这个问题


提前感谢。

如果您像现在这样直接查询mongo,是的,您将看到mongo中发生的所有文档突变。这就是像这样质疑mongo的全部目的。如果您想要mongo数据的快照视图,您需要执行mongodump之类的操作,并将bson文件放在磁盘上的某个位置(如HDFS)。否则,您将始终查询活动的、变化的数据