Database Titan graph数据库速度太慢,100000+;带索引的顶点如何优化?
以下是索引代码:Database Titan graph数据库速度太慢,100000+;带索引的顶点如何优化?,database,optimization,graph,titan,Database,Optimization,Graph,Titan,以下是索引代码: ` g = TitanFactory.build().set("storage.backend", "cassandra") .set("storage.hostname", "127.0.0.1").open(); TitanManagement mgmt = g.getManagementSystem(); PropertyKey db_local_name = mgmt.makePropertyKey("db_local_nam
`
g = TitanFactory.build().set("storage.backend", "cassandra")
.set("storage.hostname", "127.0.0.1").open();
TitanManagement mgmt = g.getManagementSystem();
PropertyKey db_local_name = mgmt.makePropertyKey("db_local_name")
.dataType(String.class).make();
mgmt.buildIndex("byDb_local_name", Vertex.class).addKey(db_local_name)
.buildCompositeIndex();
PropertyKey db_schema = mgmt.makePropertyKey("db_schema")
.dataType(String.class).make();
mgmt.buildIndex("byDb_schema", Vertex.class).addKey(db_schema)
.buildCompositeIndex();
PropertyKey db_column = mgmt.makePropertyKey("db_column")
.dataType(String.class).make();
mgmt.buildIndex("byDb_column", Vertex.class).addKey(db_column)
.buildCompositeIndex();
PropertyKey type = mgmt.makePropertyKey("type").dataType(String.class)
.make();
mgmt.buildIndex("byType", Vertex.class).addKey(type)
.buildCompositeIndex();
PropertyKey value = mgmt.makePropertyKey("value")
.dataType(Object.class).make();
mgmt.buildIndex("byValue", Vertex.class).addKey(value)
.buildCompositeIndex();
PropertyKey index = mgmt.makePropertyKey("index")
.dataType(Integer.class).make();
mgmt.buildIndex("byIndex", Vertex.class).addKey(index)
.buildCompositeIndex();
mgmt.commit();`
这里是搜索顶点,然后在3GHz 2GB RAM pc上添加带3条边的顶点。它在3小时内完成830个顶点,我有100000个数据,速度太慢了。代码如下:
for (Object[] rowObj : list) {
// TXN_ID
Iterator<Vertex> iter = g.query()
.has("db_local_name", "Report Name 1")
.has("db_schema", "MPS").has("db_column", "txn_id")
.has("value", rowObj[0]).vertices().iterator();
if (iter.hasNext()) {
vertex1 = iter.next();
logger.debug("vertex1=" + vertex1.getId() + ","
+ vertex1.getProperty("db_local_name") + ","
+ vertex1.getProperty("db_schema") + ","
+ vertex1.getProperty("db_column") + ","
+ vertex1.getProperty("type") + ","
+ vertex1.getProperty("index") + ","
+ vertex1.getProperty("value"));
}
// TXN_TYPE
iter = g.query().has("db_local_name", "Report Name 1")
.has("db_schema", "MPS").has("db_column", "txn_type")
.has("value", rowObj[1]).vertices().iterator();
if (iter.hasNext()) {
vertex2 = iter.next();
logger.debug("vertex2=" + vertex2.getId() + ","
+ vertex2.getProperty("db_local_name") + ","
+ vertex2.getProperty("db_schema") + ","
+ vertex2.getProperty("db_column") + ","
+ vertex2.getProperty("type") + ","
+ vertex2.getProperty("index") + ","
+ vertex2.getProperty("value"));
}
// WALLET_ID
iter = g.query().has("db_local_name", "Report Name 1")
.has("db_schema", "MPS").has("db_column", "wallet_id")
.has("value", rowObj[2]).vertices().iterator();
if (iter.hasNext()) {
vertex3 = iter.next();
logger.debug("vertex3=" + vertex3.getId() + ","
+ vertex3.getProperty("db_local_name") + ","
+ vertex3.getProperty("db_schema") + ","
+ vertex3.getProperty("db_column") + ","
+ vertex3.getProperty("type") + ","
+ vertex3.getProperty("index") + ","
+ vertex3.getProperty("value"));
}
vertex4 = g.addVertex(null);
vertex4.setProperty("db_local_name", "Report Name 1");
vertex4.setProperty("db_schema", "MPS");
vertex4.setProperty("db_column", "amount");
vertex4.setProperty("type", "indivisual_0");
vertex4.setProperty("value", rowObj[3].toString());
vertex4.setProperty("index", i);
vertex1.addEdge("data", vertex4);
logger.debug("vertex1 added");
vertex2.addEdge("data", vertex4);
logger.debug("vertex2 added");
vertex3.addEdge("data", vertex4);
logger.debug("vertex3 added");
i++;
g.commit();
}
(对象[]rowObj:list)的{
//TXN_ID
迭代器iter=g.query()
.has(“db\U本地名称”、“报告名称1”)
.has(“db\U模式”、“MPS”).has(“db\U列”、“txn\U id”)
.has(“value”,rowObj[0])。顶点()。迭代器();
if(iter.hasNext()){
vertex1=iter.next();
logger.debug(“vertex1=“+vertex1.getId()+”,”
+vertex1.getProperty(“db_local_name”)+,“
+vertex1.getProperty(“db_模式”)+,“
+vertex1.getProperty(“db_列”)+,“
+vertex1.getProperty(“类型”)+,“
+vertex1.getProperty(“索引”)+,“
+vertex1.getProperty(“值”);
}
//TXN_型
iter=g.query().has(“db\u local\u name”,“报告名1”)
.has(“db\U模式”、“MPS”).has(“db\U列”、“txn\U类型”)
.has(“value”,rowObj[1])。顶点()。迭代器();
if(iter.hasNext()){
vertex2=iter.next();
logger.debug(“vertex2=“+vertex2.getId()+”,”
+vertex2.getProperty(“db_local_name”)+,“
+vertex2.getProperty(“db_模式”)+,“
+vertex2.getProperty(“db_列”)+,“
+vertex2.getProperty(“类型”)+,“
+vertex2.getProperty(“索引”)+,“
+vertex2.getProperty(“值”);
}
//钱包号码
iter=g.query().has(“db\u local\u name”,“报告名1”)
.has(“db\U模式”、“MPS”).has(“db\U列”、“钱包id”)
.has(“value”,rowObj[2])。顶点()。迭代器();
if(iter.hasNext()){
vertex3=iter.next();
logger.debug(“vertex3=“+vertex3.getId()+”,”
+vertex3.getProperty(“db_local_name”)+,“
+vertex3.getProperty(“db_模式”)+,“
+vertex3.getProperty(“db_列”)+,“
+vertex3.getProperty(“类型”)+,“
+vertex3.getProperty(“索引”)+,“
+vertex3.getProperty(“值”);
}
vertex4=g.addVertex(空);
vertex4.setProperty(“db_本地_名称”、“报告名称1”);
vertex4.setProperty(“db_模式”、“MPS”);
vertex4.setProperty(“db_列”、“金额”);
vertex4.setProperty(“类型”,“个性化_0”);
vertex4.setProperty(“值”,rowObj[3].toString());
顶点4.集合属性(“索引”,i);
顶点1.添加(“数据”,顶点4);
logger.debug(“添加vertex1”);
顶点2.添加(“数据”,顶点4);
logger.debug(“添加vertex2”);
顶点3.添加(“数据”,顶点4);
logger.debug(“添加vertex3”);
i++;
g、 提交();
}
是否有优化此代码的方法?为了完整起见,Aurelius Graphs邮件列表中回答了此问题: 基本上:
mgmt.buildIndex(“by_local_name_schema_value”,Vertex.class).addKey(db_local_name).addKey(db_schema).addKey(value).buildComposite()代码>
g.commit()
,而是做一些事情
这样:if(++1%10000==0)g.commit()
存储。批量加载
如果尚未这样做BatchGraph
中所述的BatchGraph
,这样您就不必维护上面第2项中所述的事务了它仍然很慢,我无法将其投入生产。我在回答中为您添加了更多选项。830个顶点和3倍多的边对我来说没有意义。我也有性能问题。我升级到8GB的ram,与4GB相比,性能提升了400%以上。我的ram使用率接近100%,我猜这意味着它使用的是交换磁盘,因此从ram缓存上的HDD读取数据,等等,这并不好。运行一些测试,看看您的瓶颈是什么,但我猜它是ram。