C# EF生成的查询执行时间太长

C# EF生成的查询执行时间太长,c#,sql,sql-server,entity-framework,sql-server-2008-r2,C#,Sql,Sql Server,Entity Framework,Sql Server 2008 R2,我有一个由Entity Framework生成的非常简单的查询,有时当我尝试运行此查询时,执行此查询几乎需要30秒以上的时间,并且我得到了超时异常 SELECT TOP (10) [Extent1].[LinkID] AS [LinkID], [Extent1].[Title] AS [Title], [Extent1].[Url] AS [Url], [Extent1].[Description] AS [Description], [Extent1].[SentDate] AS [

我有一个由Entity Framework生成的非常简单的查询,有时当我尝试运行此查询时,执行此查询几乎需要30秒以上的时间,并且我得到了超时
异常

SELECT TOP (10) 
[Extent1].[LinkID] AS [LinkID], 
[Extent1].[Title] AS [Title], 
[Extent1].[Url] AS [Url], 
[Extent1].[Description] AS [Description], 
[Extent1].[SentDate] AS [SentDate], 
[Extent1].[VisitCount] AS [VisitCount], 
[Extent1].[RssSourceId] AS [RssSourceId], 
[Extent1].[ReviewStatus] AS [ReviewStatus], 
[Extent1].[UserAccountId] AS [UserAccountId], 
[Extent1].[CreationDate] AS [CreationDate]
FROM ( SELECT [Extent1].[LinkID] AS [LinkID], [Extent1].[Title] AS [Title], [Extent1].[Url] AS [Url], [Extent1].[Description] AS [Description], [Extent1].[SentDate] AS [SentDate], [Extent1].[VisitCount] AS [VisitCount], [Extent1].[RssSourceId] AS [RssSourceId], [Extent1].[ReviewStatus] AS [ReviewStatus], [Extent1].[UserAccountId] AS [UserAccountId], [Extent1].[CreationDate] AS [CreationDate], row_number() OVER (ORDER BY [Extent1].[SentDate] DESC) AS [row_number]
    FROM [dbo].[Links] AS [Extent1]
)  AS [Extent1]
WHERE [Extent1].[row_number] > 0
ORDER BY [Extent1].[SentDate] DESC
生成查询的代码是:

public async Task<IQueryable<TEntity>> GetAsync(Expression<Func<TEntity, bool>> filter = null,
    Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null)
{
    return await Task.Run(() =>
    {
        IQueryable<TEntity> query = _dbSet;
        if (filter != null)
        {
            query = query.Where(filter);
        }

        if (orderBy != null)
        {
            query = orderBy(query);
        }

        return query;
    });
}
任何建议都会有帮助

更新:

以下是上述代码的用法:

var dbLinks = await _uow.LinkRespository.GetAsync(filter, orderBy);
var pagedLinks = new PagedList<Link>(dbLinks, pageNumber, PAGE_SIZE);
var vmLinks = Mapper.Map<IPagedList<LinkViewItemViewModel>>(pagedLinks);

你试过用这种方法链接吗

        IQueryable<TEntity> query = _dbSet;
        return query.Where(x => (filter != null ? filter : x)
                    .Where(x => (orderBy != null ? orderBy : x));
IQueryable查询=\u dbSet;
返回查询。其中(x=>(filter!=null?filter:x)
。其中(x=>(orderBy!=null?orderBy:x));

我想知道这是否会改变EF创建的查询。

我猜
行号>0
会随着时间的推移而改变,因为您需要第2页、第3页等

因此,我很好奇它是否有助于创建此索引:

CREATE INDEX idx_links_SentDate_desc ON [dbo].[Links] ([SentDate] DESC)
诚然,如果它有效,它几乎是一个创可贴,你可能需要频繁地重建这个索引,因为我猜它会随着时间的推移而变得碎片化……
更新:检查注释!事实证明
DESC
没有任何效果,如果您的数据从低到高,应该避免使用它!

我从来没有想到您只是没有索引。经验教训-在进一步挖掘之前始终检查基础知识


如果不需要分页,那么查询可以简化为

SELECT TOP (10) 
    [Extent1].[LinkID] AS [LinkID], 
    [Extent1].[Title] AS [Title], 
    ...
FROM [dbo].[Links] AS [Extent1]
ORDER BY [Extent1].[SentDate] DESC
正如你所证实的,它运行得很快

显然,你确实需要分页,让我们看看我们能做些什么

当前版本之所以慢,是因为它首先扫描整个表,计算每行的行号,然后返回10行。我错了。SQL Server optimizer非常聪明。问题的根源在其他地方。请参阅下面的更新。


顺便说一句,正如其他人提到的,只有当
SentDate
列是唯一的时,此分页才会正确工作。如果它不是唯一的,则需要
按SentDate
排序,以及另一个唯一的列,如一些
ID
,以解决歧义

如果您不需要直接跳转到特定页面,而是始终从第1页开始,然后转到下一页、下一页等等,那么这篇优秀的文章介绍了执行此类分页的正确有效方法: 作者使用PostgreSQL进行说明,但该技术也适用于MS SQL Server。它归结为记住显示页面上最后一行的
ID
,然后在
WHERE
子句中使用该
ID
,并使用适当的支持索引检索下一页,而无需扫描之前的所有行

SQL Server 2008没有对分页的内置支持,因此我们必须使用变通方法。我将展示一种变体,它允许直接跳转到给定页面,在第一个页面上运行速度很快,但在以后的页面上运行速度会越来越慢

你的C代码中会有这些变量(
PageSize
PageNumber
),我把它们放在这里是为了说明这一点

DECLARE @VarPageSize int = 10; -- number of rows in each page
DECLARE @VarPageNumber int = 3; -- page numeration is zero-based

SELECT TOP (@VarPageSize)
    [Extent1].[LinkID] AS [LinkID]
    ,[Extent1].[Title] AS [Title]
    ,[Extent1].[Url] AS [Url]
    ,[Extent1].[Description] AS [Description]
    ,[Extent1].[SentDate] AS [SentDate]
    ,[Extent1].[VisitCount] AS [VisitCount]
    ,[Extent1].[RssSourceId] AS [RssSourceId]
    ,[Extent1].[ReviewStatus] AS [ReviewStatus]
    ,[Extent1].[UserAccountId] AS [UserAccountId]
    ,[Extent1].[CreationDate] AS [CreationDate]
FROM
    (
        SELECT TOP((@VarPageNumber + 1) * @VarPageSize)
            [Extent1].[LinkID] AS [LinkID]
            ,[Extent1].[Title] AS [Title]
            ,[Extent1].[Url] AS [Url]
            ,[Extent1].[Description] AS [Description]
            ,[Extent1].[SentDate] AS [SentDate]
            ,[Extent1].[VisitCount] AS [VisitCount]
            ,[Extent1].[RssSourceId] AS [RssSourceId]
            ,[Extent1].[ReviewStatus] AS [ReviewStatus]
            ,[Extent1].[UserAccountId] AS [UserAccountId]
            ,[Extent1].[CreationDate] AS [CreationDate]
        FROM [dbo].[Links] AS [Extent1]
        ORDER BY [Extent1].[SentDate] DESC
    ) AS [Extent1]
ORDER BY [Extent1].[SentDate] ASC
;
第一页是第1到10行,第二页是第11到20行,依此类推。 让我们看看当我们试图获取第四页时,即第31到40行时,这个查询是如何工作的。
PageSize=10
PageNumber=3
。在内部查询中,我们选择前40行。注意,我们这里不扫描整个表,我们只扫描前40行。我们甚至不需要显式的
ROW_NUMBER()
。然后我们需要从找到的40行中选择最后10行,因此外部查询选择顶部(10)与
相反的方向排序。按原样,这将以相反的顺序返回第40行到第31行。您可以在客户端上将它们重新排序为正确的顺序,或者添加一个外部查询,只需按
SentDate DESC
再次对它们进行排序。如下所示:

SELECT
    [Extent1].[LinkID] AS [LinkID]
    ,[Extent1].[Title] AS [Title]
    ,[Extent1].[Url] AS [Url]
    ,[Extent1].[Description] AS [Description]
    ,[Extent1].[SentDate] AS [SentDate]
    ,[Extent1].[VisitCount] AS [VisitCount]
    ,[Extent1].[RssSourceId] AS [RssSourceId]
    ,[Extent1].[ReviewStatus] AS [ReviewStatus]
    ,[Extent1].[UserAccountId] AS [UserAccountId]
    ,[Extent1].[CreationDate] AS [CreationDate]
FROM
    (
        SELECT TOP (@VarPageSize)
            [Extent1].[LinkID] AS [LinkID]
            ,[Extent1].[Title] AS [Title]
            ,[Extent1].[Url] AS [Url]
            ,[Extent1].[Description] AS [Description]
            ,[Extent1].[SentDate] AS [SentDate]
            ,[Extent1].[VisitCount] AS [VisitCount]
            ,[Extent1].[RssSourceId] AS [RssSourceId]
            ,[Extent1].[ReviewStatus] AS [ReviewStatus]
            ,[Extent1].[UserAccountId] AS [UserAccountId]
            ,[Extent1].[CreationDate] AS [CreationDate]
        FROM
            (
                SELECT TOP((@VarPageNumber + 1) * @VarPageSize)
                    [Extent1].[LinkID] AS [LinkID]
                    ,[Extent1].[Title] AS [Title]
                    ,[Extent1].[Url] AS [Url]
                    ,[Extent1].[Description] AS [Description]
                    ,[Extent1].[SentDate] AS [SentDate]
                    ,[Extent1].[VisitCount] AS [VisitCount]
                    ,[Extent1].[RssSourceId] AS [RssSourceId]
                    ,[Extent1].[ReviewStatus] AS [ReviewStatus]
                    ,[Extent1].[UserAccountId] AS [UserAccountId]
                    ,[Extent1].[CreationDate] AS [CreationDate]
                FROM [dbo].[Links] AS [Extent1]
                ORDER BY [Extent1].[SentDate] DESC
            ) AS [Extent1]
        ORDER BY [Extent1].[SentDate] ASC
    ) AS [Extent1]
ORDER BY [Extent1].[SentDate] DESC
此查询(作为原始查询)只有当
SentDate
是唯一的时,它才能始终正常工作。如果它不是唯一的,请在
ORDER BY
中添加唯一列。例如,如果
LinkID
是唯一的,则在最内部的查询中使用
ORDER BY SentDate DESC,LinkID DESC
。在外部查询中,颠倒顺序:
ORDER BY SentDate ASC,LinkID ASC

显然,如果您想跳转到第1000页,那么内部查询将必须读取10000行,因此越是深入,速度就越慢

在任何情况下,您都需要在
SentDate
(或
SentDate,LinkID
)上有一个索引才能使其工作。如果没有索引,查询将再次扫描整个表

我不会在这里告诉你如何将这个查询转换为EF,因为我不知道。我从来没有使用过EF。可能有一种方法。而且,显然,你可以强制它使用实际的SQL,而不是尝试使用C代码

更新 执行计划比较

在我的数据库中,我有一个包含29477859行的表
EventLogErrors
,我在SQL Server 2008上比较了EF生成的
ROW_NUMBER
查询和我在这里建议的
TOP
查询。我尝试检索第四页10行。在这两种情况下,优化器都足够聪明,可以只读取40行,如您所见从执行计划中。我使用一个主键列进行排序和分页。当我使用另一个索引列进行分页时,结果是相同的,即两个变量仅读取40行。不用说,两个变量都在几分之一秒内返回结果

SELECT TOP (10) 
[Extent1].[LinkID] AS [LinkID], 
[Extent1].[Title] AS [Title], 
.
.
.
FROM [dbo].[Links] AS [Extent1]
ORDER BY [Extent1].[SentDate] DESC
带有
顶部的变体

带有
行号的变体

这一切都意味着问题的根源在其他地方。您提到您的查询运行缓慢,只是有时运行缓慢,而我最初并没有真正注意到它。对于这种症状,我会做以下操作:

  • SELECT [Extent1].[LinkID] AS [LinkID] ,[Extent1].[Title] AS [Title] ,[Extent1].[Url] AS [Url] ,[Extent1].[Description] AS [Description] ,[Extent1].[SentDate] AS [SentDate] ,[Extent1].[VisitCount] AS [VisitCount] ,[Extent1].[RssSourceId] AS [RssSourceId] ,[Extent1].[ReviewStatus] AS [ReviewStatus] ,[Extent1].[UserAccountId] AS [UserAccountId] ,[Extent1].[CreationDate] AS [CreationDate] FROM ( SELECT TOP (@VarPageSize) [Extent1].[LinkID] AS [LinkID] ,[Extent1].[Title] AS [Title] ,[Extent1].[Url] AS [Url] ,[Extent1].[Description] AS [Description] ,[Extent1].[SentDate] AS [SentDate] ,[Extent1].[VisitCount] AS [VisitCount] ,[Extent1].[RssSourceId] AS [RssSourceId] ,[Extent1].[ReviewStatus] AS [ReviewStatus] ,[Extent1].[UserAccountId] AS [UserAccountId] ,[Extent1].[CreationDate] AS [CreationDate] FROM ( SELECT TOP((@VarPageNumber + 1) * @VarPageSize) [Extent1].[LinkID] AS [LinkID] ,[Extent1].[Title] AS [Title] ,[Extent1].[Url] AS [Url] ,[Extent1].[Description] AS [Description] ,[Extent1].[SentDate] AS [SentDate] ,[Extent1].[VisitCount] AS [VisitCount] ,[Extent1].[RssSourceId] AS [RssSourceId] ,[Extent1].[ReviewStatus] AS [ReviewStatus] ,[Extent1].[UserAccountId] AS [UserAccountId] ,[Extent1].[CreationDate] AS [CreationDate] FROM [dbo].[Links] AS [Extent1] ORDER BY [Extent1].[SentDate] DESC ) AS [Extent1] ORDER BY [Extent1].[SentDate] ASC ) AS [Extent1] ORDER BY [Extent1].[SentDate] DESC
    int takeNo = 20;
    int skipNo = 100;
    
    var results = db.ExecuteStoreQuery<Link>(
        "SELECT LinkID, Title, Url, Description, SentDate, VisitCount, RssSourceId, ReviewStatus, UserAccountId, CreationDate FROM Links", 
        null);
    
    results = results.OrderBy(x=> x.SentDate).Skip(skipNo).Take(takeNo);
    
    if (orderBy != null)
    {
        query = orderBy(query);
    }
    
    SELECT TOP (10) 
    [Extent1].[LinkID] AS [LinkID], 
    [Extent1].[Title] AS [Title], 
    [Extent1].[Url] AS [Url], 
    [Extent1].[Description] AS [Description], 
    [Extent1].[SentDate] AS [SentDate], 
    [Extent1].[VisitCount] AS [VisitCount], 
    [Extent1].[RssSourceId] AS [RssSourceId], 
    [Extent1].[ReviewStatus] AS [ReviewStatus], 
    [Extent1].[UserAccountId] AS [UserAccountId], 
    [Extent1].[CreationDate] AS [CreationDate]
    FROM [dbo].[Links] AS [Extent1]
    ORDER BY [Extent1].[SentDate] DESC
    
    select top 10 A.* from (
    SELECT * from
    [Extent1].[LinkID] AS [LinkID], 
    [Extent1].[Title] AS [Title], 
    [Extent1].[Url] AS [Url], 
    [Extent1].[Description] AS [Description], 
    [Extent1].[SentDate] AS [SentDate], 
    [Extent1].[VisitCount] AS [VisitCount], 
    [Extent1].[RssSourceId] AS [RssSourceId], 
    [Extent1].[ReviewStatus] AS [ReviewStatus], 
    [Extent1].[UserAccountId] AS [UserAccountId], 
    [Extent1].[CreationDate] AS [CreationDate]
    FROM [dbo].[Links] AS [Extent1] ) A
    ORDER BY A.[SentDate] DESC