Sql 优化-逐月查找最致力于subversion的目录
我有两个数据库。一个是带有修订号和日期的subversion日志,另一个包含修订期间更改的修订号和路径。我的查询每月查找最多提交到的目录。问题是运行需要几分钟。有人能帮我优化这个讨厌的查询吗?我相信有更好的办法Sql 优化-逐月查找最致力于subversion的目录,sql,tsql,optimization,query-optimization,greatest-n-per-group,Sql,Tsql,Optimization,Query Optimization,Greatest N Per Group,我有两个数据库。一个是带有修订号和日期的subversion日志,另一个包含修订期间更改的修订号和路径。我的查询每月查找最多提交到的目录。问题是运行需要几分钟。有人能帮我优化这个讨厌的查询吗?我相信有更好的办法 SELECT [Directory] ,[Month] ,COUNT([PathMonth]) OVER (PARTITION BY [PathMonth]) AS [Count] INTO ##temp FROM (SELECT [Path]
SELECT [Directory]
,[Month]
,COUNT([PathMonth]) OVER (PARTITION BY [PathMonth]) AS [Count] INTO ##temp
FROM
(SELECT [Path]
,[Month]
,[Directory]
,[Directory] + [Month] AS [PathMonth]
FROM
(SELECT [Path]
,SUBSTRING([Path], 0, LEN([Path]) - CHARINDEX('/', REVERSE([Path])) + 1) AS [Directory]
,CONVERT(CHAR(4), [LogDate], 120) + '-' + CONVERT(CHAR(2), [LogDate], 110) AS [Month]
FROM [SubversionLog] JOIN [PathsLog] ON [SubversionLog].[Revision] = [PathsLog].[Revision]
WHERE [Path] LIKE '/%/%/%/_%'
) one) two
ORDER BY [Month]
SELECT * INTO ##tempTwo
FROM ##temp
GROUP BY [Directory], [Month], [Count]
SELECT t1.[Directory]
,t1.[Month]
,t1.[Count]
FROM ##tempTwo t1 LEFT JOIN ##tempTwo t2 ON t1.[Month] = t2.[Month] AND t1.[Count] < t2.[Count]
WHERE t2.[Count] IS NULL
GROUP BY t1.[Directory], t1.[Month], t1.[Count]
ORDER BY [Month] DESC
IF EXISTS (SELECT * FROM ##temp)
DROP TABLE ##temp
IF EXISTS (SELECT * FROM ##tempTwo)
DROP TABLE ##tempTwo
这项工作的一半是将YYYY-MM-DD HH:MM:SS.SSS时间戳格式化为YYYY-MM,并将文件名路径转换为目录。由于这是您的版本控制,我猜表中可能没有超过二十万行。这两张表都要浏览并不是世界末日,但正如彼得·肖特(Peter Schott)所说,这将是一次很好的索引修订:
CREATE nonclustered index <name> on SubversionLog (revision)
CREATE nonclustered index <name> on PathsLog (revision)
编辑:
我认为如果不缓存一些结果,您将无法做更多的事情。您应该查看一下查询计划,看看需要优化什么。我猜是在分组、排序和/或排名中,可能是加入或键查找。对于键查找和连接,可以创建覆盖索引。对于其他内容,您需要缓存结果。我不会缓存一个真正的表,因为这意味着需要一个额外的表来保持最新。相反,我将使用物化视图,以便SQLServer为您保持更新。当然,这意味着更新速度会慢一些,但对于源代码管理日志,每分钟只更新几次,我不认为这会是一个大问题。我将首先查看查询中的SubversionLog连接路径slog部分。修订是否索引?有多少路径不符合该格式,因为该查找将进行某种扫描。如果你有前几个月,在最近一个月过滤掉,以限制读取的行-前几个月的日期不应该改变,对吗?将所有这些存储在一个永久表中以供参考,您将能够比较现有月份和新添加的月份,而无需重新分析所有内容。哇,非常感谢。您让我的查询快了20倍。@sirdank比几分钟快了20倍意味着查询仍然需要几秒钟以上,对吗?我又添加了一些建议,可以将查询时间缩短到不到一秒钟。
;with filteredData as (
SELECT [path],
Substring([path], 0, Len([path]) - Charindex('/',
Reverse([path])) + 1)
AS
[Directory],
CONVERT(CHAR(4), [logdate], 120) + '-'
+ CONVERT(CHAR(2), [logdate], 110)
AS [Month]
FROM [subversionlog]
JOIN [pathslog]
ON [subversionlog].[revision] = [pathslog].[revision]
WHERE [path] LIKE '/%/%/%/_%'
), countRevisions as (
SELECT [month],
[directory],
count(*) as [Count]
FROM filteredData
GROUP BY [MONTH], [directory]
), rankDirectories as (
SELECT *, RANK() over (partition by month order by count desc) as [Rank]
from countRevisions
)
select [month], [directory], [count]
from rankDirectories
WHERE [rank] = 1