Fiware Cygnus总是希望在收到通知时创建一个新的MongoDB集合

Fiware Cygnus总是希望在收到通知时创建一个新的MongoDB集合,fiware,fiware-cygnus,Fiware,Fiware Cygnus,我想使用Fiware Cygnus将历史数据保存到MongoDB中。但是,每当Cygnus收到通知时,它都希望创建一个新集合,而不是在集合已经存在的情况下添加文档。我从天鹅座得到了这个错误信息: time=2017-01-14T20:58:11.785Z | lvl=WARN | corr=28eca6a8-da9c-11e6-841b-0242ac120009 | trans=9fa2d345-9aa1-4ff6-84df-75de5829a449 | srv=itg | subsrv=/bu

我想使用Fiware Cygnus将历史数据保存到MongoDB中。但是,每当Cygnus收到通知时,它都希望创建一个新集合,而不是在集合已经存在的情况下添加文档。我从天鹅座得到了这个错误信息:

time=2017-01-14T20:58:11.785Z | lvl=WARN | corr=28eca6a8-da9c-11e6-841b-0242ac120009 | trans=9fa2d345-9aa1-4ff6-84df-75de5829a449 | srv=itg | subsrv=/building1 | comp=cygnus-ngsi | op=processNewBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[590] : Command failed with error 48: 'a collection 'sth_itg.sth_/building1_TemperatureRoom1_room' already exists' on server iot-mongo:27017. The full response is { "ok" : 0.0, "errmsg" : "a collection 'sth_itg.sth_/building1_TemperatureRoom1_room' already exists", "code" : 48, "codeName" : "NamespaceExists" }
这是我的agent.conf文件:

cygnus-ngsi.sources = http-source
cygnus-ngsi.sinks = mongo-sink
cygnus-ngsi.channels = mongo-channel

cygnus-ngsi.sources.http-source.type = org.apache.flume.source.http.HTTPSource
cygnus-ngsi.sources.http-source.channels = mongo-channel
cygnus-ngsi.sources.http-source.port = 5050
cygnus-ngsi.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.NGSIRestHandler
cygnus-ngsi.sources.http-source.handler.notification_target = /notify
cygnus-ngsi.sources.http-source.handler.default_service = default
cygnus-ngsi.sources.http-source.handler.default_service_path = /
cygnus-ngsi.sources.http-source.interceptors = ts gi
cygnus-ngsi.sources.http-source.interceptors.ts.type = timestamp
cygnus-ngsi.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor$Builder
cygnus-ngsi.sources.http-source.interceptors.gi.grouping_rules_conf_file = /opt/apache-flume/conf/grouping_rules.conf


cygnus-ngsi.sinks.mongo-sink.type = com.telefonica.iot.cygnus.sinks.NGSIMongoSink
cygnus-ngsi.sinks.mongo-sink.channel = mongo-channel
#cygnus-ngsi.sinks.mongo-sink.enable_encoding = false
#cygnus-ngsi.sinks.mongo-sink.enable_grouping = false
#cygnus-ngsi.sinks.mongo-sink.enable_name_mappings = false
#cygnus-ngsi.sinks.mongo-sink.enable_lowercase = false
#cygnus-ngsi.sinks.mongo-sink.data_model = dm-by-entity
#cygnus-ngsi.sinks.mongo-sink.attr_persistence = row
cygnus-ngsi.sinks.mongo-sink.mongo_hosts = iot-mongo:27017
cygnus-ngsi.sinks.mongo-sink.mongo_username =
cygnus-ngsi.sinks.mongo-sink.mongo_password =
#cygnus-ngsi.sinks.mongo-sink.db_prefix = sth_
#cygnus-ngsi.sinks.mongo-sink.collection_prefix = sth_
#cygnus-ngsi.sinks.mongo-sink.batch_size = 1
#cygnus-ngsi.sinks.mongo-sink.batch_timeout = 30
#cygnus-ngsi.sinks.mongo-sink.batch_ttl = 10
#cygnus-ngsi.sinks.mongo-sink.data_expiration = 0
#cygnus-ngsi.sinks.mongo-sink.collections_size = 0
#cygnus-ngsi.sinks.mongo-sink.max_documents = 0
#cygnus-ngsi.sinks.mongo-sink.ignore_white_spaces = true

cygnus-ngsi.channels.mongo-channel.type = com.telefonica.iot.cygnus.channels.CygnusMemoryChannel
cygnus-ngsi.channels.mongo-channel.capacity = 1000
cygnus-ngsi.channels.mongo-channel.transactionCapacity = 100
我做错什么了吗?有人能帮忙吗


提前谢谢。

天鹅座总是在收到通知后,在插入数据之前尝试创建集合。为什么?因为请求MongoDB存在集合以及仅在集合不存在时请求MongoDB创建集合是通过驱动程序执行的两个操作;要求MongoDB始终创建集合只是一个操作。因此,如果该集合存在,那么如果您看到调试日志跟踪告诉您该集合存在,则应该没有问题。(*)

问题是您遇到了错误的警告日志跟踪。您使用的是哪个版本的MongoDB?查看代码后,在创建集合时解释MongoDB响应时可能会出现问题,这不是基于错误代码,而是基于消息字符串:如果MongoDB返回的消息与预期不同,则会引发错误。当然,这是必须改进的


(*)这可以通过使用已创建数据库和集合的“缓存”来改进。这样的缓存可以通过MongoDB驱动程序节省大量操作。这个想法已经在CKAN和MySQL接收器上实现了。

您如何解释一下您的尝试?最好创建一个。在猎户座中,我创建了一个订阅,当温度值发生变化时,它将向天鹅座发送通知。尽管Cygnus能够将数据保存到MySQL中,但由于我试图在上面解释的问题,它无法将数据保存到MongoDB。问题很简单,Cygnus总是想在MongoDB中创建一个新集合,而不是向其中添加一个新文档
try {
   backend.createDatabase(dbName);
   backend.createCollection(dbName, collectionName, collectionsSize, maxDocuments, dataExpiration);
   backend.insertContextDataRaw(dbName, collectionName, aggregation);
} catch (Exception e) {
   throw new CygnusPersistenceError("-, " + e.getMessage());
} // try catch
// create the collection
try {
   db.createCollection(collectionName);
} catch (Exception e) {
   if (e.getMessage().contains("collection already exists")) {
      LOGGER.debug("Collection already exists, nothing to create");
   } else {
      throw e;
   } // if else
} // try catch