使用“将json加载到Neo4j中”;apoc.load.json";:不完全数据检索
我试图从mongodb集合中检索json数据,并在neo4j中将其创建为标签。我正在使用MongoRESTAPI从mongodb检索json。我接着是威廉·里昂的《MongoDB和Neo4j的多语言持久性》 问题是我无法加载所有数据。以下是json数据的格式使用“将json加载到Neo4j中”;apoc.load.json";:不完全数据检索,neo4j,cypher,neo4j-apoc,Neo4j,Cypher,Neo4j Apoc,我试图从mongodb集合中检索json数据,并在neo4j中将其创建为标签。我正在使用MongoRESTAPI从mongodb检索json。我接着是威廉·里昂的《MongoDB和Neo4j的多语言持久性》 问题是我无法加载所有数据。以下是json数据的格式 > db.cves.findOne() { "_id" : ObjectId("5a37226550eb46004dea39b0"), "vulnerable_configuration_cpe_2_2
> db.cves.findOne()
{
"_id" : ObjectId("5a37226550eb46004dea39b0"),
"vulnerable_configuration_cpe_2_2" : [
"cpe:/o:bsdi:bsd_os:3.1",
"cpe:/o:freebsd:freebsd:1.0",
"cpe:/o:freebsd:freebsd:1.1",
"cpe:/o:freebsd:freebsd:1.1.5.1",
"cpe:/o:freebsd:freebsd:1.2",
"cpe:/o:freebsd:freebsd:2.0",
"cpe:/o:freebsd:freebsd:2.0.1",
"cpe:/o:freebsd:freebsd:2.0.5",
"cpe:/o:freebsd:freebsd:2.1.5",
"cpe:/o:freebsd:freebsd:2.1.6",
"cpe:/o:freebsd:freebsd:2.1.6.1",
"cpe:/o:freebsd:freebsd:2.1.7",
"cpe:/o:freebsd:freebsd:2.1.7.1",
"cpe:/o:freebsd:freebsd:2.2",
"cpe:/o:freebsd:freebsd:2.2.2",
"cpe:/o:freebsd:freebsd:2.2.3",
"cpe:/o:freebsd:freebsd:2.2.4",
"cpe:/o:freebsd:freebsd:2.2.5",
"cpe:/o:freebsd:freebsd:2.2.6",
"cpe:/o:freebsd:freebsd:2.2.8",
"cpe:/o:freebsd:freebsd:3.0",
"cpe:/o:openbsd:openbsd:2.3",
"cpe:/o:openbsd:openbsd:2.4"
],
"impact" : {
"integrity" : "NONE",
"availability" : "PARTIAL",
"confidentiality" : "NONE"
},
"vulnerable_configuration" : [
"cpe:2.3:o:bsdi:bsd_os:3.1",
"cpe:2.3:o:freebsd:freebsd:1.0",
"cpe:2.3:o:freebsd:freebsd:1.1",
"cpe:2.3:o:freebsd:freebsd:1.1.5.1",
"cpe:2.3:o:freebsd:freebsd:1.2",
"cpe:2.3:o:freebsd:freebsd:2.0",
"cpe:2.3:o:freebsd:freebsd:2.0.1",
"cpe:2.3:o:freebsd:freebsd:2.0.5",
"cpe:2.3:o:freebsd:freebsd:2.1.5",
"cpe:2.3:o:freebsd:freebsd:2.1.6",
"cpe:2.3:o:freebsd:freebsd:2.1.6.1",
"cpe:2.3:o:freebsd:freebsd:2.1.7",
"cpe:2.3:o:freebsd:freebsd:2.1.7.1",
"cpe:2.3:o:freebsd:freebsd:2.2",
"cpe:2.3:o:freebsd:freebsd:2.2.2",
"cpe:2.3:o:freebsd:freebsd:2.2.3",
"cpe:2.3:o:freebsd:freebsd:2.2.4",
"cpe:2.3:o:freebsd:freebsd:2.2.5",
"cpe:2.3:o:freebsd:freebsd:2.2.6",
"cpe:2.3:o:freebsd:freebsd:2.2.8",
"cpe:2.3:o:freebsd:freebsd:3.0",
"cpe:2.3:o:openbsd:openbsd:2.3",
"cpe:2.3:o:openbsd:openbsd:2.4"
],
"cvss" : 5,
"references" : [
"http://www.openbsd.org/errata23.html#tcpfix"
],
"Modified" : ISODate("2010-12-16T00:00:00Z"),
"summary" : "ip_input.c in BSD-derived TCP/IP implementations allows remote attackers to cause a denial of service (crash or hang) via crafted packets.",
"cwe" : "CWE-20",
"Published" : ISODate("1999-12-30T00:00:00Z"),
"cvss-time" : ISODate("2004-01-01T00:00:00Z"),
"access" : {
"vector" : "NETWORK",
"authentication" : "NONE",
"complexity" : "LOW"
},
"id" : "CVE-1999-0001"
}
>
下面的查询工作正常
//Load CPE collection from CVEDB of mongodb as CVE Label
CALL apoc.load.json('http://127.0.0.1:28017/cvedb/cves/') YIELD value
UNWIND value.rows as cveData
MERGE (c:CVE{_id:cveData._id['$oid']})
ON Create set c.id=cveData.id, c.cvss = cveData.cvss
输出:
Added 1000 labels, created 1000 nodes, set 2970 properties, completed after 462 ms.
问题:
> db.cves.count();
99022
集合中有99022记录,为什么在我的密码查询中只创建1000个节点而不是99022
感谢仅创建1000个节点的一些可能原因:
MERGE
将不会创建新节点。也许你已经有了一些匹配的节点cveData.\u id['$oid']
值,则这些CVE中最多有一个将创建一个新节点CALL apoc.load.json('http://127.0.0.1:28017/cvedb/cves/') YIELD value
RETURN count(*);
CALL apoc.load.json('http://127.0.0.1:28017/cvedb/cves/') YIELD value
UNWIND value.rows as cveData
RETURN count(*);
CALL apoc.load.json('http://127.0.0.1:28017/cvedb/cves/') YIELD value
UNWIND value.rows as cveData
RETURN count(distinct cveData._id['$oid']);
也
亲爱的迈克尔,谢谢你的回复。下面是来自cypher shell
neo4j>调用apoc.load.json的响应http://172.22.0.2:28017/cvedb/cves/“)收益值返回计数(*);”输出:
1`neo4j>调用apoc.load.json('http://172.22.0.2:28017/cvedb/cves/“)作为cveData返回计数(*)的屈服值展开值.rows代码>输出:1000
调用apoc.load.json('http://172.22.0.2:28017/cvedb/cves/“)作为cveData返回计数的收益值展开value.rows(不同的cveData._id[“$oid]”)代码>输出:1000
调用apoc.load.json('http://172.22.0.2:28017/cvedb/cves/“)屈服值释放值。行作为cveData返回cveData.\u id LIMIT 1代码>输出:{
$oid:“5a37226550eb46004dea39b0”}
如果您需要,我可以与您共享docker compose、cvedb和README。我不明白为什么APOC调用没有检索到超过1000条记录的数据。我通过使用以下命令#mongoexport--db cvedb--collection cves--out cve-cvedb.json--jsonArray--pretty
从mongodb手动导出集合,解决了这个问题。mongoexport--db cvedb--collection cves--out cve-cve-cvedb.json--jsonArray--pretty
,然后将它放入下面使用的/import密码命令,WITH“file:///cve-cvedb.json“as url调用apoc.load.json(url)产生值返回计数(*)代码>输出为99167
亲爱的cybersam,感谢您的回复。以下是mongodb中一些计数查询的响应<代码>>db.cpe.count()代码>输出:225354
,>db.cwe.count()代码>输出:722
,>db.cves.count()代码>输出:99167
。下面是使用APCO对neo4j中的一些计数查询的响应<代码>调用apoc.load.json('http://172.22.0.3:28017/cvedb/cpe/“)作为cpeData返回计数(*)的产量值展开值.rows代码>输出:1000
,调用apoc.load.json('http://172.22.0.3:28017/cvedb/cwe/“)作为cweData返回计数(*)的屈服值展开值.rows代码>输出:722
,并调用apoc.load.json('http://172.22.0.3:28017/cvedb/cves/“)作为cvesData返回计数(*)的收益值展开value.rows代码>输出:1000
如果您愿意,我可以与您分享docker compose、cvedb和README。我不明白为什么APOC调用没有检索到超过1000条记录的数据。我通过使用以下命令#mongoexport--db cvedb--collection cves--out cve-cvedb.json--jsonArray--pretty
从mongodb手动导出集合,解决了这个问题。mongoexport--db cvedb--collection cves--out cve-cve-cvedb.json--jsonArray--pretty
,然后将它放入下面使用的/import密码命令,WITH“file:///cve-cvedb.json“as url调用apoc.load.json(url)产生值返回计数(*)代码>输出为99167
CALL apoc.load.json('http://127.0.0.1:28017/cvedb/cves/') YIELD value
UNWIND value.rows as cveData
RETURN cveData._id LIMIT 1;