Database Titan graph数据库速度太慢,100000+;带索引的顶点如何优化?

Database Titan graph数据库速度太慢,100000+;带索引的顶点如何优化?,database,optimization,graph,titan,Database,Optimization,Graph,Titan,以下是索引代码: ` g = TitanFactory.build().set("storage.backend", "cassandra") .set("storage.hostname", "127.0.0.1").open(); TitanManagement mgmt = g.getManagementSystem(); PropertyKey db_local_name = mgmt.makePropertyKey("db_local_nam

以下是索引代码:

`
g = TitanFactory.build().set("storage.backend", "cassandra")
            .set("storage.hostname", "127.0.0.1").open();

    TitanManagement mgmt = g.getManagementSystem();

    PropertyKey db_local_name = mgmt.makePropertyKey("db_local_name")
            .dataType(String.class).make();
    mgmt.buildIndex("byDb_local_name", Vertex.class).addKey(db_local_name)
            .buildCompositeIndex();

    PropertyKey db_schema = mgmt.makePropertyKey("db_schema")
            .dataType(String.class).make();
    mgmt.buildIndex("byDb_schema", Vertex.class).addKey(db_schema)
            .buildCompositeIndex();

    PropertyKey db_column = mgmt.makePropertyKey("db_column")
            .dataType(String.class).make();
    mgmt.buildIndex("byDb_column", Vertex.class).addKey(db_column)
            .buildCompositeIndex();

    PropertyKey type = mgmt.makePropertyKey("type").dataType(String.class)
            .make();
    mgmt.buildIndex("byType", Vertex.class).addKey(type)
            .buildCompositeIndex();

    PropertyKey value = mgmt.makePropertyKey("value")
            .dataType(Object.class).make();
    mgmt.buildIndex("byValue", Vertex.class).addKey(value)
            .buildCompositeIndex();

    PropertyKey index = mgmt.makePropertyKey("index")
            .dataType(Integer.class).make();
    mgmt.buildIndex("byIndex", Vertex.class).addKey(index)
            .buildCompositeIndex();

    mgmt.commit();`
这里是搜索顶点,然后在3GHz 2GB RAM pc上添加带3条边的顶点。它在3小时内完成830个顶点,我有100000个数据,速度太慢了。代码如下:

for (Object[] rowObj : list) {
            // TXN_ID
            Iterator<Vertex> iter = g.query()
                    .has("db_local_name", "Report Name 1")
                    .has("db_schema", "MPS").has("db_column", "txn_id")
                    .has("value", rowObj[0]).vertices().iterator();
            if (iter.hasNext()) {
                vertex1 = iter.next();
                logger.debug("vertex1=" + vertex1.getId() + ","
                        + vertex1.getProperty("db_local_name") + ","
                        + vertex1.getProperty("db_schema") + ","
                        + vertex1.getProperty("db_column") + ","
                        + vertex1.getProperty("type") + ","
                        + vertex1.getProperty("index") + ","
                        + vertex1.getProperty("value"));
            }
            // TXN_TYPE
            iter = g.query().has("db_local_name", "Report Name 1")
                    .has("db_schema", "MPS").has("db_column", "txn_type")
                    .has("value", rowObj[1]).vertices().iterator();
            if (iter.hasNext()) {
                vertex2 = iter.next();
                logger.debug("vertex2=" + vertex2.getId() + ","
                        + vertex2.getProperty("db_local_name") + ","
                        + vertex2.getProperty("db_schema") + ","
                        + vertex2.getProperty("db_column") + ","
                        + vertex2.getProperty("type") + ","
                        + vertex2.getProperty("index") + ","
                        + vertex2.getProperty("value"));
            }
            // WALLET_ID
            iter = g.query().has("db_local_name", "Report Name 1")
                    .has("db_schema", "MPS").has("db_column", "wallet_id")
                    .has("value", rowObj[2]).vertices().iterator();
            if (iter.hasNext()) {
                vertex3 = iter.next();
                logger.debug("vertex3=" + vertex3.getId() + ","
                        + vertex3.getProperty("db_local_name") + ","
                        + vertex3.getProperty("db_schema") + ","
                        + vertex3.getProperty("db_column") + ","
                        + vertex3.getProperty("type") + ","
                        + vertex3.getProperty("index") + ","
                        + vertex3.getProperty("value"));
            }

            vertex4 = g.addVertex(null);
            vertex4.setProperty("db_local_name", "Report Name 1");
            vertex4.setProperty("db_schema", "MPS");
            vertex4.setProperty("db_column", "amount");
            vertex4.setProperty("type", "indivisual_0");
            vertex4.setProperty("value", rowObj[3].toString());
            vertex4.setProperty("index", i);

            vertex1.addEdge("data", vertex4);
            logger.debug("vertex1 added");
            vertex2.addEdge("data", vertex4);
            logger.debug("vertex2 added");
            vertex3.addEdge("data", vertex4);
            logger.debug("vertex3 added");
            i++;
            g.commit();
        }
(对象[]rowObj:list)的
{
//TXN_ID
迭代器iter=g.query()
.has(“db\U本地名称”、“报告名称1”)
.has(“db\U模式”、“MPS”).has(“db\U列”、“txn\U id”)
.has(“value”,rowObj[0])。顶点()。迭代器();
if(iter.hasNext()){
vertex1=iter.next();
logger.debug(“vertex1=“+vertex1.getId()+”,”
+vertex1.getProperty(“db_local_name”)+,“
+vertex1.getProperty(“db_模式”)+,“
+vertex1.getProperty(“db_列”)+,“
+vertex1.getProperty(“类型”)+,“
+vertex1.getProperty(“索引”)+,“
+vertex1.getProperty(“值”);
}
//TXN_型
iter=g.query().has(“db\u local\u name”,“报告名1”)
.has(“db\U模式”、“MPS”).has(“db\U列”、“txn\U类型”)
.has(“value”,rowObj[1])。顶点()。迭代器();
if(iter.hasNext()){
vertex2=iter.next();
logger.debug(“vertex2=“+vertex2.getId()+”,”
+vertex2.getProperty(“db_local_name”)+,“
+vertex2.getProperty(“db_模式”)+,“
+vertex2.getProperty(“db_列”)+,“
+vertex2.getProperty(“类型”)+,“
+vertex2.getProperty(“索引”)+,“
+vertex2.getProperty(“值”);
}
//钱包号码
iter=g.query().has(“db\u local\u name”,“报告名1”)
.has(“db\U模式”、“MPS”).has(“db\U列”、“钱包id”)
.has(“value”,rowObj[2])。顶点()。迭代器();
if(iter.hasNext()){
vertex3=iter.next();
logger.debug(“vertex3=“+vertex3.getId()+”,”
+vertex3.getProperty(“db_local_name”)+,“
+vertex3.getProperty(“db_模式”)+,“
+vertex3.getProperty(“db_列”)+,“
+vertex3.getProperty(“类型”)+,“
+vertex3.getProperty(“索引”)+,“
+vertex3.getProperty(“值”);
}
vertex4=g.addVertex(空);
vertex4.setProperty(“db_本地_名称”、“报告名称1”);
vertex4.setProperty(“db_模式”、“MPS”);
vertex4.setProperty(“db_列”、“金额”);
vertex4.setProperty(“类型”,“个性化_0”);
vertex4.setProperty(“值”,rowObj[3].toString());
顶点4.集合属性(“索引”,i);
顶点1.添加(“数据”,顶点4);
logger.debug(“添加vertex1”);
顶点2.添加(“数据”,顶点4);
logger.debug(“添加vertex2”);
顶点3.添加(“数据”,顶点4);
logger.debug(“添加vertex3”);
i++;
g、 提交();
}

是否有优化此代码的方法?

为了完整起见,Aurelius Graphs邮件列表中回答了此问题:

基本上:

  • 构建/使用真实的复合索引:
    mgmt.buildIndex(“by_local_name_schema_value”,Vertex.class).addKey(db_local_name).addKey(db_schema).addKey(value).buildComposite()
  • 不要在每个循环周期后调用
    g.commit()
    ,而是做一些事情 这样:
    if(++1%10000==0)g.commit()
  • 打开
    存储。批量加载
    如果尚未这样做
  • 如果你能在卡桑德拉扔的是2G的RAM,考虑使用BelkLayDB。卡桑德拉的RAM最小,可能会想要“更多”
  • 我不知道您的数据的性质,但您是否可以对其进行预排序,并使用博客文章和-使用
    BatchGraph
    中所述的
    BatchGraph
    ,这样您就不必维护上面第2项中所述的事务了

  • 它仍然很慢,我无法将其投入生产。我在回答中为您添加了更多选项。830个顶点和3倍多的边对我来说没有意义。我也有性能问题。我升级到8GB的ram,与4GB相比,性能提升了400%以上。我的ram使用率接近100%,我猜这意味着它使用的是交换磁盘,因此从ram缓存上的HDD读取数据,等等,这并不好。运行一些测试,看看您的瓶颈是什么,但我猜它是ram。