SQL死锁-高流量
我有一个大约需要15秒才能执行的存储过程。当数以百计的请求进来时，我们会看到无法接受的页面加载时间。有时两分钟。页面根据ID加载结果，因此每个人的结果都不相同 我的解决方案是使用一个暂存表，只有在页面5分钟未加载时才更新它。我认为这会减少存储过程的负载。但是现在我看到了这个暂存台想法的问题 如果页面已有5分钟未被点击，则根据ID从暂存表中删除，然后运行存储过程并将结果插入暂存表 然后使用总结果计数更新临时表，根据ID从临时表中选择 正如您所看到的，它正在执行删除、插入、更新和选择操作。在负载下，我会遇到很多死锁 有几个问题： 返回昂贵的结果集以简单地显示在网页上的最佳实践是什么？ 如何改进暂存表方法并确保没有死锁？ 代码：SQL死锁-高流量,sql,sql-server,stored-procedures,query-optimization,Sql,Sql Server,Stored Procedures,Query Optimization,我有一个大约需要15秒才能执行的存储过程。当数以百计的请求进来时，我们会看到无法接受的页面加载时间。有时两分钟。页面根据ID加载结果，因此每个人的结果都不相同 我的解决方案是使用一个暂存表，只有在页面5分钟未加载时才更新它。我认为这会减少存储过程的负载。但是现在我看到了这个暂存台想法的问题 如果页面已有5分钟未被点击，则根据ID从暂存表中删除，然后运行存储过程并将结果插入暂存表 然后使用总结果计数更新临时表，根据ID从临时表中选择 正如您所看到的，它正在执行删除、插入、更新和选择操作。在负载下，
根据获取实际数据所需的时间，缓存数据的想法可能确实是值得的；问题在于在Web服务器层而不是在数据库层这样做有多容易 这就是说，既然您已经得到了这一部分，那么让我们看看存储过程 首先是一些一般性意见： 我真的，真的不喜欢读未经承诺的书。我知道你添加此选项是为了避免锁定，但肮脏阅读的想法确实让我感到紧张。您可能会像这样向来电者返回一半或错误的信息。就我个人而言，我宁愿等一段时间好的信息，也不愿等坏的信息。 不要使用SELECT COUNT*。。。以确定是否有任何记录。服务器需要扫描所有相关记录才能获得号码，但您真正感兴趣的是是否有号码。在存在的地方使用。 更妙的是，既然你在事后选择了TOP 1，那么就用它的结果吧。如果它找到一个值，那么一定有一条记录！ ISNULL@LastUpdated，=在使用datetime字段/变量时有点可疑。相反，对于此检查，使用@LastUpdated为空 您执行删除操作，然后可能是一个繁重的插入操作。我想这需要一段时间，这就是僵局的根源。正确的锁定应该有帮助，但我不知道它如何配合上面的readuncommitted。 插入后，再次检查新插入的数据，以确定有多少行。因为这里唯一相关的行是我们刚刚插入的行，所以我们可以放心地假设@ROWCOUNT已经包含了答案。读取该系统变量不需要任何努力。正在进行选择计数*。。。对系统来说是非常非常繁重的 就我个人而言，我更喜欢将INSERT分成两部分，并使用temp表来保存中间结果。从理论上讲，我不确定这会对性能产生多大影响，因为服务器需要做更多的工作，但我认为这会带来更少的惊喜，特别是如果您在两者之间添加一些统计数据更新。只有测试才能真正说明问题。 使用temp表的副作用是，您可以简单地从所述表返回结果，而不需要返回缓存表，从而避免与其他进程的交互。 我还建议使用动态sql，因为它为排序提供了灵活性。“RowNumber”中的外壳结构确实有效，但我怀疑它是否过于高效。 不管怎样，至于锁。之所以会出现死锁，是因为您在同一记录上执行了大量操作，并且可能同时来自不同的连接。因此，一个连接可能正在执行插入操作，而另一个连接尝试执行删除或更新等操作。。那可不好。 为了避免这种情况，我会使用锁定，但即使在那里，我也可能会遇到问题，因此我会尝试使用一个应用程序锁定，当另一个应用程序锁定仍在刷新缓存表中的数据时，“暂停”任何连接。更重要的是，我们可以将此锁定“限制”到正在处理的@EventId 对我在下面提出的代码做一些快速而肮脏的更改。可能会有一些语法错误，甚至可能会有一些逻辑错误，因此您可能需要根据需要调整/修复。但我希望它能为你指明正确的方向
CREATE procedure [dbo].[BB02_ListFundraisersForEvent] (
@DesignEventId int,
@Offset int,
@PageSize int,
@SearchTerms varchar(100) = null,
@OrderByField varchar(25) = 'DEFAULT',
@OrderByDirection varchar(5) = 'ASC'
)
-- exec BB02_ListFundraisersForEvent 38639, 0, 10, '', '', 'ASC', null
as
-- set transaction isolation level read uncommitted
declare @UpdateIncrement DateTime = DATEADD(MINUTE, -5, GETDATE());
declare @FundraiserCount int;
declare @LastUpdated DateTime;
declare @PAGE_STATUS_CANCELED int;
declare @TOTAL_TYPE_NON_REJECTED int;
declare @TOTAL_TYPE_REGISTRATION int;
declare @PROFILE_APPEAL_WEB_DIR_FAMILY int;
declare @PROFILE_LEVEL_WEB_DIR_FAMILY int;
declare @TotalCount int
declare @cache_was_updated bit
declare @sql nvarchar(max)
declare @rc int
declare @LockName sysname
set @TOTAL_TYPE_NON_REJECTED = 2;
set @TOTAL_TYPE_REGISTRATION = 3;
set @PAGE_STATUS_CANCELED = 3
set @PROFILE_APPEAL_WEB_DIR_FAMILY = 3;
set @PROFILE_LEVEL_WEB_DIR_FAMILY = 2;
if @OrderByField not in ('FirstName', 'LastName', 'TotalRaised') set @OrderByField = 'DEFAULT';
IF @OrderByDirection not in ('ASC', 'DESC') set @OrderByDirection = 'ASC';
SET @cache_was_updated = 0
IF isnull(@SearchTerms, '') = ''
BEGIN
select @LastUpdated = NULL
select TOP 1 @LastUpdated = lastupdated from bb02_olr_getsupporterscache where designeventid = @DesignEventId
IF( (@LastUpdated IS NULL) -- no value found means no data present for given @DesignEventId
OR (@LastUpdated < @UpdateIncrement ) ) -- or value found, but too far in the past)
BEGIN
-- prepare new batch
set @LockName = 'CacheUpdate' + Convert(nvarchar(100), @DesignEventId)
-- get exclusive applock on this @DesignEventId
-- => we should be the only ones that can update this!
EXEC @rc = sp_getapplock @Resource = @LockName,
@LockMode = 'Exlusive',
@LockTimeout = 0, -- if another process already has the lock, we will skip the update
@lockOwner = 'Session'
IF @rc > 0
BEGIN
-- lock obtained
-- => let's do the cache update
SET @TotalCount = 0
-- fundraising pages
SELECT
egg.EventGivingGroupName as AppealName,
awd.WebDirectoryName as AppealWebDirectory,
c.FirstName,
egg.ImageChoice,
c.LastName,
egg.PhotoUrl,
cwd.WebDirectoryName as ProfileWebDirectory,
eggt.TotalRaisedOffline,
eggt.TotalRaisedOnline,
eggt.TotalContributions,
CAST(egg.DisplayPhoto AS bit) AS DisplayPhoto,
CAST(CASE WHEN ISNULL(dei.DesignEventId, 0) != 0 then 1 else 0 end as bit) as HasStockImages
INTO #staging
FROM
BB02_Event e
INNER JOIN
BB02_EventFundraiserRevenueStream efrs on e.EventId = efrs.EventId
INNER JOIN
BB02_EventGivingGroup egg on efrs.EventFundraiserRevenueStreamId = egg.EventFundraiserRevenueStreamId
INNER JOIN
BB02_EventGivingGroupTotal eggt on egg.EventGivingGroupId = eggt.EventGivingGroupId
INNER JOIN
BB02_Consumer c on c.ConsumerId = egg.ConsumerId
INNER JOIN
BB02_WebDirectory cwd on cwd.WebDirectoryId = c.DefaultWebDirectoryId and cwd.WebDirectoryFamilyId = @PROFILE_LEVEL_WEB_DIR_FAMILY
INNER JOIN
BB02_WebDirectory awd on awd.EventGivingGroupId = egg.EventGivingGroupId and awd.WebDirectoryFamilyId = @PROFILE_APPEAL_WEB_DIR_FAMILY
inner join BB02_DesignEvent de on e.EventId = de.EventId and egg.DesignId = de.DesignId
left join (select distinct DesignEventId from BB02_DesignEventImage) dei on de.DesignEventId = dei.DesignEventId
where eggt.EventGivingGroupTotalTypeId =
case when de.AddFeesToTotal = 1 then @TOTAL_TYPE_REGISTRATION -- 3 includes registration fees
else @TOTAL_TYPE_NON_REJECTED /* 1 = Confirmed, 2 = Not Rejected */
end
and egg.Status <> @PAGE_STATUS_CANCELED
and de.DesignEventId = @DesignEventId
and egg.IsDeleted = 0
SET @TotalCount = @TotalCount + @@ROWCOUNT
-- registrants without pages
INSERT #staging ( AppealName,
AppealWebDirectory,
FirstName,
ImageChoice,
LastName,
PhotoURL,
ProfileWebDirectory,
TotalRaisedOffline,
TotalRaisedOnline,
TotalContributions,
DisplayPhoto,
HasStockImages,
LastUpdated)
select
'' as AppealName,
'' as AppealWebDirectory,
FirstName,
'' as ImageChoice,
LastName,
'' as PhotoURL,
'' as ProfileWebDirectory,
0 as TotalRaisedOffline,
0 as TotalRaisedOnline,
0 as TotalContributions,
'' as DisplayPhoto,
cast(case when isnull(dei.DesignEventId, 0) != 0 then 1 else 0 end as bit) as HasStockImages
from BB02_ConsumerEventRegistration cer
left join (select distinct DesignEventId from BB02_DesignEventImage) dei on cer.DesignEventId = dei.DesignEventId
where cer.DesignEventId = @DesignEventId
and cer.ConsumerId not in (
select egg.ConsumerId
from BB02_EventGivingGroup egg
inner join BB02_EventFundraiserRevenueStream efrs on efrs.EventFundraiserRevenueStreamId = egg.EventFundraiserRevenueStreamId
inner join BB02_DesignEvent de on efrs.EventId = de.EventId and egg.DesignId = de.DesignId
where de.DesignEventId = @DesignEventId
)
SET @TotalCount = @TotalCount + @@ROWCOUNT
-- out with the old, in with the new
BEGIN TRANSACTION
DELETE FROM BB02_OLR_GetSupportersCache WITH (XLOCK, ROWLOCK, HOLDLOCK)
WHERE designeventid = @DesignEventId
INSERT INTO bb02_olr_getsupporterscache (DesignEventId,
AppealName,
AppealWebDirectory,
FirstName,
ImageChoice,
LastName,
PhotoURL,
ProfileWebDirectory,
TotalRaisedOffline,
TotalRaisedOnline,
TotalContributions,
DisplayPhoto,
HasStockImages,
LastUpdated,
TotalCount)
SELECT DesignEventId = @DesignEventId,
AppealName,
AppealWebDirectory,
FirstName,
ImageChoice,
LastName,
PhotoURL,
ProfileWebDirectory,
TotalRaisedOffline,
TotalRaisedOnline,
TotalContributions,
DisplayPhoto,
HasStockImages,
LastUpdated = CURRENT_TIMESTAMP,
TotalCount = @TotalCount
COMMIT TRANSAcTION
-- don't drop #staging table yet, we'll re-use it for output again right away
SELECT @cache_was_updated = 1
END
ELSE
BEGIN
-- no lock was obtained, simply wait for the lock to be freed as that will
-- be the moment the new data comes available
EXEC @rc = sp_getapplock @Resource = @LockName,
@LockMode = 'Exclusive',
@LockTimeout = 600000, -- 10 minutes should be enough, end-user will be pretty annyoyed anyway =)
@lockOwner = 'Session'
END
-- free the lock agian, we assume reading locks are handled properly
EXEC sp_releaseapplock @Resource = @LockName
END -- cache update required or not?
-- fetch results
SET @sql = Convert(nvarchar(max), N'')
+ N'
SELECT * FROM (
select
TotalCount' + (CASE WHEN @cache_was_updated = 1 THEN ' = @TotalCount' ELSE N'' END) + ',
SupporterId,
AppealName,
AppealWebDirectory,
FirstName,
ImageChoice,
LastName,
PhotoURL,
ProfileWebDirectory,
TotalRaisedOffline,
TotalRaisedOnline,
TotalContributions,
DisplayPhoto,
HasStockImages,
row_number() over (order by ' +
case when @OrderByField IN ('FirstName', 'LastName') then @OrderByField
when @OrderByField = 'TotalRaised' then '(TotalRaisedOnline + TotalRaisedOffline)'
else 'AppealName' end
+ ' ' + @OrderByDirection + ') as rownumber
from ' + (CASE WHEN @cache_was_updated = 1 THEN '#staging' ELSE 'bb02_olr_getsupporterscache where designeventid = @DesignEventId' END) + '
) q
where q.rownumber > @Offset
and q.rownumber <= @Offset + @PageSize
order by rownumber;'
exec sp_executesql @stmt = @sql,
@params = N'@TotalCount int, @DesignEventId int, @Offset int, @PageSize int',
@TotalCount = @TotalCount,
@DesignEventId = @DesignEventId,
@Offset = @Offset,
@PageSize = @PageSize
DROP TABLE #staging
END
-- other part not looked at (yet) as I guess the trouble comes from the refresh, and that's only in the first part as far as I can tell ...
END
我还可以提一个建议吗。在您的过程中使用SET NOCOUNT ON。这将提高性能，而且很容易获得：请参阅仔细查看FROM子句中项目的顺序。考虑放置子查询来强制查询编译器一个方向或另一个方向。如果其中一个内部连接块比其他块更积极地减少结果集，那么将该块移近FROM将产生好处 奥斯。我没有看到在这个过程中有任何事务开始-整个调用是否被事务包围？您应该用必要的列名替换所有select*。第一次检查@FundraiserCount和@LastUpdated可替换为单个顶部。注意，top1 select中缺少ORDER BY子句。并将Totalcount更新移到该事务之外。再说一次，直到你发现什么是真正的僵局-你无法修复它。
CREATE procedure [dbo].[BB02_ListFundraisersForEvent] (
@DesignEventId int,
@Offset int,
@PageSize int,
@SearchTerms varchar(100) = null,
@OrderByField varchar(25) = 'DEFAULT',
@OrderByDirection varchar(5) = 'ASC'
)
-- exec BB02_ListFundraisersForEvent 38639, 0, 10, '', '', 'ASC', null
as
-- set transaction isolation level read uncommitted
declare @UpdateIncrement DateTime = DATEADD(MINUTE, -5, GETDATE());
declare @FundraiserCount int;
declare @LastUpdated DateTime;
declare @PAGE_STATUS_CANCELED int;
declare @TOTAL_TYPE_NON_REJECTED int;
declare @TOTAL_TYPE_REGISTRATION int;
declare @PROFILE_APPEAL_WEB_DIR_FAMILY int;
declare @PROFILE_LEVEL_WEB_DIR_FAMILY int;
declare @TotalCount int
declare @cache_was_updated bit
declare @sql nvarchar(max)
declare @rc int
declare @LockName sysname
set @TOTAL_TYPE_NON_REJECTED = 2;
set @TOTAL_TYPE_REGISTRATION = 3;
set @PAGE_STATUS_CANCELED = 3
set @PROFILE_APPEAL_WEB_DIR_FAMILY = 3;
set @PROFILE_LEVEL_WEB_DIR_FAMILY = 2;
if @OrderByField not in ('FirstName', 'LastName', 'TotalRaised') set @OrderByField = 'DEFAULT';
IF @OrderByDirection not in ('ASC', 'DESC') set @OrderByDirection = 'ASC';
SET @cache_was_updated = 0
IF isnull(@SearchTerms, '') = ''
BEGIN
select @LastUpdated = NULL
select TOP 1 @LastUpdated = lastupdated from bb02_olr_getsupporterscache where designeventid = @DesignEventId
IF( (@LastUpdated IS NULL) -- no value found means no data present for given @DesignEventId
OR (@LastUpdated < @UpdateIncrement ) ) -- or value found, but too far in the past)
BEGIN
-- prepare new batch
set @LockName = 'CacheUpdate' + Convert(nvarchar(100), @DesignEventId)
-- get exclusive applock on this @DesignEventId
-- => we should be the only ones that can update this!
EXEC @rc = sp_getapplock @Resource = @LockName,
@LockMode = 'Exlusive',
@LockTimeout = 0, -- if another process already has the lock, we will skip the update
@lockOwner = 'Session'
IF @rc > 0
BEGIN
-- lock obtained
-- => let's do the cache update
SET @TotalCount = 0
-- fundraising pages
SELECT
egg.EventGivingGroupName as AppealName,
awd.WebDirectoryName as AppealWebDirectory,
c.FirstName,
egg.ImageChoice,
c.LastName,
egg.PhotoUrl,
cwd.WebDirectoryName as ProfileWebDirectory,
eggt.TotalRaisedOffline,
eggt.TotalRaisedOnline,
eggt.TotalContributions,
CAST(egg.DisplayPhoto AS bit) AS DisplayPhoto,
CAST(CASE WHEN ISNULL(dei.DesignEventId, 0) != 0 then 1 else 0 end as bit) as HasStockImages
INTO #staging
FROM
BB02_Event e
INNER JOIN
BB02_EventFundraiserRevenueStream efrs on e.EventId = efrs.EventId
INNER JOIN
BB02_EventGivingGroup egg on efrs.EventFundraiserRevenueStreamId = egg.EventFundraiserRevenueStreamId
INNER JOIN
BB02_EventGivingGroupTotal eggt on egg.EventGivingGroupId = eggt.EventGivingGroupId
INNER JOIN
BB02_Consumer c on c.ConsumerId = egg.ConsumerId
INNER JOIN
BB02_WebDirectory cwd on cwd.WebDirectoryId = c.DefaultWebDirectoryId and cwd.WebDirectoryFamilyId = @PROFILE_LEVEL_WEB_DIR_FAMILY
INNER JOIN
BB02_WebDirectory awd on awd.EventGivingGroupId = egg.EventGivingGroupId and awd.WebDirectoryFamilyId = @PROFILE_APPEAL_WEB_DIR_FAMILY
inner join BB02_DesignEvent de on e.EventId = de.EventId and egg.DesignId = de.DesignId
left join (select distinct DesignEventId from BB02_DesignEventImage) dei on de.DesignEventId = dei.DesignEventId
where eggt.EventGivingGroupTotalTypeId =
case when de.AddFeesToTotal = 1 then @TOTAL_TYPE_REGISTRATION -- 3 includes registration fees
else @TOTAL_TYPE_NON_REJECTED /* 1 = Confirmed, 2 = Not Rejected */
end
and egg.Status <> @PAGE_STATUS_CANCELED
and de.DesignEventId = @DesignEventId
and egg.IsDeleted = 0
SET @TotalCount = @TotalCount + @@ROWCOUNT
-- registrants without pages
INSERT #staging ( AppealName,
AppealWebDirectory,
FirstName,
ImageChoice,
LastName,
PhotoURL,
ProfileWebDirectory,
TotalRaisedOffline,
TotalRaisedOnline,
TotalContributions,
DisplayPhoto,
HasStockImages,
LastUpdated)
select
'' as AppealName,
'' as AppealWebDirectory,
FirstName,
'' as ImageChoice,
LastName,
'' as PhotoURL,
'' as ProfileWebDirectory,
0 as TotalRaisedOffline,
0 as TotalRaisedOnline,
0 as TotalContributions,
'' as DisplayPhoto,
cast(case when isnull(dei.DesignEventId, 0) != 0 then 1 else 0 end as bit) as HasStockImages
from BB02_ConsumerEventRegistration cer
left join (select distinct DesignEventId from BB02_DesignEventImage) dei on cer.DesignEventId = dei.DesignEventId
where cer.DesignEventId = @DesignEventId
and cer.ConsumerId not in (
select egg.ConsumerId
from BB02_EventGivingGroup egg
inner join BB02_EventFundraiserRevenueStream efrs on efrs.EventFundraiserRevenueStreamId = egg.EventFundraiserRevenueStreamId
inner join BB02_DesignEvent de on efrs.EventId = de.EventId and egg.DesignId = de.DesignId
where de.DesignEventId = @DesignEventId
)
SET @TotalCount = @TotalCount + @@ROWCOUNT
-- out with the old, in with the new
BEGIN TRANSACTION
DELETE FROM BB02_OLR_GetSupportersCache WITH (XLOCK, ROWLOCK, HOLDLOCK)
WHERE designeventid = @DesignEventId
INSERT INTO bb02_olr_getsupporterscache (DesignEventId,
AppealName,
AppealWebDirectory,
FirstName,
ImageChoice,
LastName,
PhotoURL,
ProfileWebDirectory,
TotalRaisedOffline,
TotalRaisedOnline,
TotalContributions,
DisplayPhoto,
HasStockImages,
LastUpdated,
TotalCount)
SELECT DesignEventId = @DesignEventId,
AppealName,
AppealWebDirectory,
FirstName,
ImageChoice,
LastName,
PhotoURL,
ProfileWebDirectory,
TotalRaisedOffline,
TotalRaisedOnline,
TotalContributions,
DisplayPhoto,
HasStockImages,
LastUpdated = CURRENT_TIMESTAMP,
TotalCount = @TotalCount
COMMIT TRANSAcTION
-- don't drop #staging table yet, we'll re-use it for output again right away
SELECT @cache_was_updated = 1
END
ELSE
BEGIN
-- no lock was obtained, simply wait for the lock to be freed as that will
-- be the moment the new data comes available
EXEC @rc = sp_getapplock @Resource = @LockName,
@LockMode = 'Exclusive',
@LockTimeout = 600000, -- 10 minutes should be enough, end-user will be pretty annyoyed anyway =)
@lockOwner = 'Session'
END
-- free the lock agian, we assume reading locks are handled properly
EXEC sp_releaseapplock @Resource = @LockName
END -- cache update required or not?
-- fetch results
SET @sql = Convert(nvarchar(max), N'')
+ N'
SELECT * FROM (
select
TotalCount' + (CASE WHEN @cache_was_updated = 1 THEN ' = @TotalCount' ELSE N'' END) + ',
SupporterId,
AppealName,
AppealWebDirectory,
FirstName,
ImageChoice,
LastName,
PhotoURL,
ProfileWebDirectory,
TotalRaisedOffline,
TotalRaisedOnline,
TotalContributions,
DisplayPhoto,
HasStockImages,
row_number() over (order by ' +
case when @OrderByField IN ('FirstName', 'LastName') then @OrderByField
when @OrderByField = 'TotalRaised' then '(TotalRaisedOnline + TotalRaisedOffline)'
else 'AppealName' end
+ ' ' + @OrderByDirection + ') as rownumber
from ' + (CASE WHEN @cache_was_updated = 1 THEN '#staging' ELSE 'bb02_olr_getsupporterscache where designeventid = @DesignEventId' END) + '
) q
where q.rownumber > @Offset
and q.rownumber <= @Offset + @PageSize
order by rownumber;'
exec sp_executesql @stmt = @sql,
@params = N'@TotalCount int, @DesignEventId int, @Offset int, @PageSize int',
@TotalCount = @TotalCount,
@DesignEventId = @DesignEventId,
@Offset = @Offset,
@PageSize = @PageSize
DROP TABLE #staging
END
-- other part not looked at (yet) as I guess the trouble comes from the refresh, and that's only in the first part as far as I can tell ...
END