Performance 如何使用大表提高sqlite的性能?
SQlite数据库,具有单个表和60000000条记录。运行简单查询的时间超过100秒。 我曾尝试切换到postgeSQL,但它的性能更差。 没有在mySQL或msSQL上测试它。 Shell我拆分了表,比如说每个pointID有不同的表-有几百个?或者每个月有不同的表-那么我最多有10000000条记录 sql方案:Performance 如何使用大表提高sqlite的性能?,performance,sqlite,Performance,Sqlite,SQlite数据库,具有单个表和60000000条记录。运行简单查询的时间超过100秒。 我曾尝试切换到postgeSQL,但它的性能更差。 没有在mySQL或msSQL上测试它。 Shell我拆分了表,比如说每个pointID有不同的表-有几百个?或者每个月有不同的表-那么我最多有10000000条记录 sql方案: CREATE TABLE `collectedData` ( `id` INTEGER, `timeStamp` double, `timeDate
CREATE TABLE `collectedData` (
`id` INTEGER,
`timeStamp` double,
`timeDateStr` nvarchar,
`pointID` nvarchar,
`pointIDindex` double,
`trendNumber` integer,
`status` nvarchar,
`value` double,
PRIMARY KEY(`id`)
);
CREATE INDEX `idx_pointID` ON `collectedData` (
`pointID`
);
CREATE INDEX `idx_pointIDindex` ON `collectedData` (
`pointIDindex`
);
CREATE INDEX `idx_timeStamp` ON `collectedData` (
`timeStamp`
);
CREATE INDEX `idx_trendNumber` ON `collectedData` (
`trendNumber`
);
下一次查询耗时107秒:
select * from collectedData
where
trendNumber =1
and status <> '' and
timestamp <=1556793244
and pointid in ('point1','point2','pont3','point4','point5','point6','point7','point8','point9','pointa')
and pointIDindex % 1 = 0
order by timestamp desc, id desc limit 5000
性能提高了3倍
编辑2:by@Raymond Nijland offer:执行计划为:
使用覆盖索引idx_All trendNumber=?搜索表收集的数据?和pointID=?
0执行列表子查询1
0使用临时B树作为订购依据
多亏了他-使用这些数据,我将查询中规则的顺序更改为下一个:
select * from (
select * from collectedData
where
trendNumber =1
and status <> '' and
timestamp <=1556793244
and pointid in ('point1','point2','pont3','point4','point5','point6','point7','point8','point9','pointa')
and pointIDindex % 1 = 0
order by id desc limit 5000
) order by timestamp desc
这对我来说有了很大的改善,问题解决了。在@RaymondNijland让我检查执行计划后,我将查询更改为:
select * from (
select * from collectedData
where
trendNumber =1
and status <> '' and
timestamp <=1556793244
and pointid in ('point1','point2','pont3','point4','point5','point6','point7','point8','point9','pointa')
and pointIDindex % 1 = 0
order by id desc limit 5000
) order by timestamp desc
此查询的结果与另一个查询相同,但排序前减少记录数的速度不是120倍。有关性能的问题还应包括问题中涉及的每个查询的一些结果。此外,您无法比较PostgreSQL和SQLite的性能,因为它们解决的问题完全不同。。因为PostgreSQL是一个服务器/客户端应用程序,而SQLite是一个文件嵌入式数据库应用程序。谢谢@RaymondNijland-这帮助我解决了这个问题,对查询做了一点修改。
select * from (
select * from collectedData
where
trendNumber =1
and status <> '' and
timestamp <=1556793244
and pointid in ('point1','point2','pont3','point4','point5','point6','point7','point8','point9','pointa')
and pointIDindex % 1 = 0
order by id desc limit 5000
) order by timestamp desc
select * from (
select * from collectedData
where
trendNumber =1
and status <> '' and
timestamp <=1556793244
and pointid in ('point1','point2','pont3','point4','point5','point6','point7','point8','point9','pointa')
and pointIDindex % 1 = 0
order by id desc limit 5000
) order by timestamp desc