Sql server 提高sqlserver中insert的速度
我正在尝试在SQLServer2008中优化以下查询。执行计划显示此查询相对于批处理的百分比为90%。插入显示75%。我正在寻找一个选项,做批量插入,并发插入等有人抛出加快进程的灯。我可以执行Sql server 提高sqlserver中insert的速度,sql-server,sql-server-2008,Sql Server,Sql Server 2008,我正在尝试在SQLServer2008中优化以下查询。执行计划显示此查询相对于批处理的百分比为90%。插入显示75%。我正在寻找一个选项,做批量插入,并发插入等有人抛出加快进程的灯。我可以执行SqlBulkCopy或使用OpenRowset功能吗?我已经确保where条件下的所有字段都有索引 INSERT INTO [xxxxxxx].[dbo].[xxxxxxx] WITH (Tablock) SELECT s.companyId, ti.ticker
SqlBulkCopy
或使用OpenRowset
功能吗?我已经确保where条件下的所有字段都有索引
INSERT INTO [xxxxxxx].[dbo].[xxxxxxx] WITH (Tablock)
SELECT
s.companyId,
ti.tickerSymbol,
s.securityName,
ex.exchangeName,
cur.currencyName,
primaryFlag = CASE
WHEN ti.primaryFlag = 1 AND s.primaryFlag = 1
THEN 1
ELSE 0
END,
ti.tradingItemId,
peq.pricingDate, peq.priceOpen,
peq.priceHigh, peq.priceLow, peq.priceMid,
peq.priceClose, peq.priceBid, peq.priceAsk,
peq.volume, peq.adjustmentFactor, peq.VWAP,
mc.marketCap, mc.TEV, mc.sharesOutstanding
FROM
ciqsecurity s
INNER JOIN
CoreReferenceStaging.dbo.MarketDataTemp1 a ON a.companyId = s.companyId
INNER JOIN
ciqtradingitem ti ON s.securityid = ti.securityid
LEFT JOIN
ciqpriceequity peq ON peq.tradingitemid = ti.tradingitemid
LEFT JOIN
CoreReferenceStaging.dbo.MarketDataTemp2 mc ON (mc.companyId= s.companyId AND mc.pricingDate = peq.pricingDate)
INNER JOIN
ciqExchange ex ON ex.exchangeId = ti. exchangeId
INNER JOIN
ciqCurrency cur ON cur.currencyid = ti.currencyId
ORDER BY
peq.pricingDate DESC
我试着将插入物切成几批,但似乎需要更长的时间。例如:
DECLARE @BatchSize int = 1000
WHILE 1 = 1
BEGIN
INSERT INTO [xxxxxx].[dbo].[xxxxxxx] with (Tablock)
select
s.companyId,
ti.tickerSymbol,
s.securityName,
ex.exchangeName,
cur.currencyName,
primaryFlag = case when ti.primaryFlag = 1 and s.primaryFlag = 1 then 1 else 0 end,
ti.tradingItemId,
peq.pricingDate,
peq.priceOpen,
peq.priceHigh,
peq.priceLow,
peq.priceMid,
peq.priceClose,
peq.priceBid,
peq.priceAsk,
peq.volume,
peq.adjustmentFactor,
peq.VWAP,
mc.marketCap,
mc.TEV,
mc.sharesOutstanding
from ciqsecurity s
inner join CoreReferenceStaging.dbo.MarketDataTemp1 a on a.companyId = s.companyId
inner join ciqtradingitem ti on s.securityid = ti.securityid
left join ciqpriceequity peq on peq.tradingitemid = ti.tradingitemid
left join CoreReferenceStaging.dbo.MarketDataTemp2 mc on (mc.companyId= s.companyId and mc.pricingDate = peq.pricingDate)
inner join ciqExchange ex on ex.exchangeId = ti. exchangeId
inner join ciqCurrency cur on cur.currencyid = ti.currencyId
order by peq.pricingDate desc
IF @@ROWCOUNT < @BatchSize BREAK END
执行计划1
执行计划2
查询似乎经过了很好的调整,似乎没有需要返工的关键点。 它只需提取超过4000-5000万条记录,只需11秒即可执行。在我看来,你再也不能改进这个问题了。。。
您可以对索引碎片进行一些检查,以防重新生成它们,但您不会在5秒之内。。。我认为查询没有问题,也没有瓶颈……查询似乎经过了很好的调整,似乎没有需要返工的关键点。 它只需提取超过4000-5000万条记录,只需11秒即可执行。在我看来,你再也不能改进这个问题了。。。
您可以对索引碎片进行一些检查,以防重新生成它们,但您不会在5秒之内。。。我认为查询没有问题,没有瓶颈…索引会减慢插入速度。你需要所有你创建的索引吗?等等。。。插入零件速度慢,还是选择速度慢?加快插入速度是一项挑战。加快选择以将所有记录插入内存…瓶颈在哪里。select上的索引是否不在目标表上?正在禁用日志记录/恢复。。。事务系统的所有开销都会导致插入速度变慢。插入是否需要order by?最简单的解决方案是,如果目标表具有索引,则可以在继续插入记录之前删除索引。然后你可以重新创建索引Hanks Darion我觉得相同的索引会减慢插入速度。你需要所有你创建的索引吗?等等。。。插入零件速度慢,还是选择速度慢?加快插入速度是一项挑战。加快选择以将所有记录插入内存…瓶颈在哪里。select上的索引是否不在目标表上?正在禁用日志记录/恢复。。。事务系统的所有开销都会导致插入速度变慢。插入是否需要order by?最简单的解决方案是,如果目标表具有索引,则可以在继续插入记录之前删除索引。然后你可以重新创建indexesThanks Darion我也有同感
Table 'ciqExchange'. Scan count 17, logical reads 67, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'ciqCurrency'. Scan count 1, logical reads 4, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'MarketDataTemp1'. Scan count 10, logical reads 4, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'ciqSecurity'. Scan count 500, logical reads 53703, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'ciqTradingItem'. Scan count 17, logical reads 33082, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'Worktable'. Scan count 11382, logical reads 1193682, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'ciqPriceEquity'. Scan count 15681, logical reads 96425, physical reads 0, read-ahead reads 3, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'MarketDataTemp2'. Scan count 17, logical reads 1648, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'Worktable'. Scan count 0, logical reads 0, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.