Performance 如何使用大表提高sqlite的性能?

Performance 如何使用大表提高sqlite的性能?,performance,sqlite,Performance,Sqlite,SQlite数据库,具有单个表和60000000条记录。运行简单查询的时间超过100秒。 我曾尝试切换到postgeSQL,但它的性能更差。 没有在mySQL或msSQL上测试它。 Shell我拆分了表,比如说每个pointID有不同的表-有几百个?或者每个月有不同的表-那么我最多有10000000条记录 sql方案: CREATE TABLE `collectedData` ( `id` INTEGER, `timeStamp` double, `timeDate

SQlite数据库,具有单个表和60000000条记录。运行简单查询的时间超过100秒。 我曾尝试切换到postgeSQL,但它的性能更差。 没有在mySQL或msSQL上测试它。 Shell我拆分了表,比如说每个pointID有不同的表-有几百个?或者每个月有不同的表-那么我最多有10000000条记录

sql方案:

CREATE TABLE `collectedData` (
    `id`    INTEGER,
    `timeStamp` double,
    `timeDateStr`   nvarchar,
    `pointID`   nvarchar,
    `pointIDindex`  double,
    `trendNumber`   integer,
    `status`    nvarchar,
    `value` double,
    PRIMARY KEY(`id`)
);

CREATE INDEX `idx_pointID` ON `collectedData` (
    `pointID`
);

CREATE INDEX `idx_pointIDindex` ON `collectedData` (
    `pointIDindex`
);

CREATE INDEX `idx_timeStamp` ON `collectedData` (
    `timeStamp`
);

CREATE INDEX `idx_trendNumber` ON `collectedData` (
    `trendNumber`
);
下一次查询耗时107秒:

select * from collectedData 
where 
trendNumber =1 
and status <> ''  and 
timestamp <=1556793244 
and pointid in ('point1','point2','pont3','point4','point5','point6','point7','point8','point9','pointa') 
and pointIDindex % 1 = 0  
order by timestamp desc, id desc limit 5000
性能提高了3倍

编辑2:by@Raymond Nijland offer:执行计划为: 使用覆盖索引idx_All trendNumber=?搜索表收集的数据?和pointID=? 0执行列表子查询1 0使用临时B树作为订购依据

多亏了他-使用这些数据,我将查询中规则的顺序更改为下一个:

select * from (
select * from collectedData 
where 
trendNumber =1 
and status <> ''  and 
timestamp <=1556793244 
and pointid in ('point1','point2','pont3','point4','point5','point6','point7','point8','point9','pointa') 

and pointIDindex % 1 = 0  
order by id desc limit 5000
) order by timestamp desc

这对我来说有了很大的改善,问题解决了。

在@RaymondNijland让我检查执行计划后,我将查询更改为:

select * from (
select * from collectedData 
where 
trendNumber =1 
and status <> ''  and 
timestamp <=1556793244 
and pointid in ('point1','point2','pont3','point4','point5','point6','point7','point8','point9','pointa') 

and pointIDindex % 1 = 0  
order by id desc limit 5000
) order by timestamp desc

此查询的结果与另一个查询相同,但排序前减少记录数的速度不是120倍。

有关性能的问题还应包括问题中涉及的每个查询的一些结果。此外,您无法比较PostgreSQL和SQLite的性能,因为它们解决的问题完全不同。。因为PostgreSQL是一个服务器/客户端应用程序,而SQLite是一个文件嵌入式数据库应用程序。谢谢@RaymondNijland-这帮助我解决了这个问题,对查询做了一点修改。
select * from (
select * from collectedData 
where 
trendNumber =1 
and status <> ''  and 
timestamp <=1556793244 
and pointid in ('point1','point2','pont3','point4','point5','point6','point7','point8','point9','pointa') 

and pointIDindex % 1 = 0  
order by id desc limit 5000
) order by timestamp desc
select * from (
select * from collectedData 
where 
trendNumber =1 
and status <> ''  and 
timestamp <=1556793244 
and pointid in ('point1','point2','pont3','point4','point5','point6','point7','point8','point9','pointa') 

and pointIDindex % 1 = 0  
order by id desc limit 5000
) order by timestamp desc