表上带条件的SQL完全联接
我有两个版本的相同的数据集,我需要做一个完整的连接,以找到其中一个丢失的记录,两者都有一些丢失的记录。我想出了两种方法,但都有缺点。我的数据集大小和筛选条件非常大 解决方案1使用CTE有一个缺点,这将拆分过滤器,使代码更难阅读,我只想有一个查询:表上带条件的SQL完全联接,sql,sql-server,join,jointable,Sql,Sql Server,Join,Jointable,我有两个版本的相同的数据集,我需要做一个完整的连接,以找到其中一个丢失的记录,两者都有一些丢失的记录。我想出了两种方法,但都有缺点。我的数据集大小和筛选条件非常大 解决方案1使用CTE有一个缺点,这将拆分过滤器,使代码更难阅读,我只想有一个查询: create table #temp (id int, vers nvarchar(1)) insert into #temp select 1,'a' union select 2,'a' union select 3,'a'
create table #temp (id int, vers nvarchar(1))
insert into #temp select 1,'a' union select 2,'a' union select 3,'a'
union select 1,'b' union select 2,'b' union select 100,'b'
;WITH vers_a as (SELECT * FROM #temp WHERE vers = 'a')
,vers_b as (SELECT * FROM #temp WHERE vers = 'b')
SELECT ta.id, tb.id, ta.vers, tb.vers
FROM vers_a ta
FULL JOIN vers_b tb on ta.id = tb.id
WHERE ta.id is null or tb.id is null
drop table #temp
解决方案2复制了过滤器,且执行计划更大:
create table #temp (id int, vers nvarchar(1))
insert into #temp select 1,'a' union select 2,'a' union select 3,'a'
union select 1,'b' union select 2,'b' union select 100,'b'
SELECT ta.id, tb.id, ta.vers, tb.vers
FROM #temp ta
FULL JOIN #temp tb on ta.id = tb.id and ta.vers = 'a' and tb.vers = 'b'
WHERE (ta.id is null or tb.id is null) and (ta.vers = 'a' or tb.vers = 'b')
drop table #temp
所以我的问题是,有没有可能像解决方案2那样,没有双条件定义,有更小的执行计划,比如解决方案1
编辑:在一个查询中运行两个解决方案时,我可以看到解决方案2的成本为26%,而解决方案1的成本为45%,尽管它的执行计划较小。我希望更快的解决方案不必像我在问题中所说的那样有更小的执行计划,如果可能的话,不需要代码重复
Edit2:很抱歉误导了第1次编辑,我不擅长优化:我在~1.5 mil行集上测试了这一点,解决方案1更快,要使用该集,请执行以下操作:
create table #temp (id int, vers nvarchar(1))
insert into #temp select 1,'a' union select 2,'a' union select 3,'a'
union select 1,'b' union select 2,'b' union select 100,'b'
while (select count(*) from #temp) < 1000000
begin
insert into #temp select id+ABS(CHECKSUM(NewId()))%10000, vers from #temp
end
这应该有一个好的计划。对vers的索引可能会有所帮助
SELECT ta.id, tb.id, ta.vers, tb.vers
FROM (SELECT * FROM #temp WHERE vers = 'a') ta
FULL JOIN (SELECT * FROM #temp WHERE vers = 'b') tb on ta.id = tb.id
WHERE (ta.id is null or tb.id is null)
编辑
做了一些测试。与其他两个版本相比,上面的查询具有更好的CPU
-- SETUP
drop table temp;
go
create table temp (
id int
,vers nvarchar(1));
insert temp(id,vers)
select top(100000)
row_number() over(order by (select null)) / 2
, case ABS(CHECKSUM(NewId())) % 2 when 0 then 'a' else 'b' end
from sys.all_objects t, sys.all_objects t1 ;
create index idx_temp_vers on temp(vers) include(id)
with
fillfactor=90;
select top(50) *
from temp;
-- TEST RUNS
SET STATISTICS TIME ON;
print ' 1 index query 1 '
SELECT ta.id, tb.id, ta.vers, tb.vers
FROM (SELECT * FROM temp WHERE vers = 'a') ta
FULL JOIN (SELECT * FROM temp WHERE vers = 'b') tb on ta.id = tb.id
WHERE (ta.id is null or tb.id is null)
;
print ' 1 index query 2 '
SELECT ta.id, tb.id, ta.vers, tb.vers
FROM temp ta
FULL JOIN temp tb on ta.id = tb.id and ta.vers = 'a' and tb.vers = 'b'
WHERE (ta.id is null or tb.id is null) and (ta.vers = 'a' or tb.vers = 'b')
;
print ' 1 index query 3 '
SELECT ta.id, TA.vers
from temp ta
where ta.vers = 'a'
and TA.id NOT IN(SELECT tb.id FROM temp tb WHERE tb.vers = 'b')
UNION ALL
SELECT tb.id, Tb.vers
from temp tb
where tb.vers = 'b'
and Tb.id NOT IN(SELECT ta.id FROM temp ta WHERE ta.vers = 'a')
-- One more index
create index idx_temp_id on temp(id)
with
fillfactor=90;
print ' 2 indexes query 1 '
SELECT ta.id, tb.id, ta.vers, tb.vers
FROM (SELECT * FROM temp WHERE vers = 'a') ta
FULL JOIN (SELECT * FROM temp WHERE vers = 'b') tb on ta.id = tb.id
WHERE (ta.id is null or tb.id is null)
;
print ' 2 indexes query 2 '
SELECT ta.id, tb.id, ta.vers, tb.vers
FROM temp ta
FULL JOIN temp tb on ta.id = tb.id and ta.vers = 'a' and tb.vers = 'b'
WHERE (ta.id is null or tb.id is null) and (ta.vers = 'a' or tb.vers = 'b')
;
print ' 2 indexes query 3 '
SELECT ta.id, TA.vers
from temp ta
where ta.vers = 'a'
and TA.id NOT IN(SELECT tb.id FROM temp tb WHERE tb.vers = 'b')
UNION ALL
SELECT tb.id, Tb.vers
from temp tb
where tb.vers = 'b'
and Tb.id NOT IN(SELECT ta.id FROM temp ta WHERE ta.vers = 'a')
SET STATISTICS TIME OFF;
结果
1 index query 1
(49898 row(s) affected)
SQL Server Execution Times:
CPU time = 156 ms, elapsed time = 3825 ms.
1 index query 2
(49898 row(s) affected)
SQL Server Execution Times:
CPU time = 281 ms, elapsed time = 2962 ms.
1 index query 3
(49898 row(s) affected)
SQL Server Execution Times:
CPU time = 422 ms, elapsed time = 2508 ms.
2 indexes query 1
(49898 row(s) affected)
SQL Server Execution Times:
CPU time = 172 ms, elapsed time = 2679 ms.
2 indexes query 2
(49898 row(s) affected)
SQL Server Execution Times:
CPU time = 406 ms, elapsed time = 3468 ms.
2 indexes query 3
(49898 row(s) affected)
SQL Server Execution Times:
CPU time = 407 ms, elapsed time = 3728 ms.
这样可以避免返回包含空值的列吗
SELECT ta.id, TA.vers from #temp ta
where ta.vers = 'a'
and TA.id NOT IN(SELECT tb.id FROM #temp tb WHERE tb.vers = 'b')
UNION ALL
SELECT tb.id, Tb.vers from #temp tb
where tb.vers = 'b'
and Tb.id NOT IN(SELECT ta.id FROM #temp ta WHERE ta.vers = 'a')
你可以试试这两个问题,也许?您的预期输出是什么?预期输出是任何解决方案的结果,除了我理解的两个查询1个完整-要排除什么,它也是slow@AndrewDeighton在以下示例中,id不是唯一的OP@thinkeye谢谢我现在明白了,我已经用解决方案2运行了它,我得到了与解决方案1相同的45%的查询成本,基本上相同的查询,CTE移到了查询,解决方案2得到了26%。我不知道为什么:查看测试结果谢谢,我也得到了几乎相同的结果,我想为了更快的查询,我将不得不接受更难阅读的代码。这对于小数据集是可以的,但是对于大数据集,不在其中的数据集效率非常低,我使用约150万行进行了测试,并在运行5分钟后取消。