Postgresql索引列的排序速度较慢
我有一个很大的疑问,那就是最后我必须按照一些标准来排序。查询的主表大约有150万行,我有很多JOIN语句。当我运行查询时需要9分钟以上,当运行EXPLAIN SELECT时,很明显是“ORDER BY”子句需要很多时间。ORDER BY语句后面的列使用btree索引,并且是BIGINT。有人能建议我如何缩短执行时间吗?下面是解释分析选择的一部分。。。(需要9分钟以上才能完成) 查询:Postgresql索引列的排序速度较慢,sql,postgresql,Sql,Postgresql,我有一个很大的疑问,那就是最后我必须按照一些标准来排序。查询的主表大约有150万行,我有很多JOIN语句。当我运行查询时需要9分钟以上,当运行EXPLAIN SELECT时,很明显是“ORDER BY”子句需要很多时间。ORDER BY语句后面的列使用btree索引,并且是BIGINT。有人能建议我如何缩短执行时间吗?下面是解释分析选择的一部分。。。(需要9分钟以上才能完成) 查询: explain ANALYSE select aa2.aid as cid ,aa2.id ,aa2.ty ,
explain ANALYSE
select aa2.aid as cid
,aa2.id ,aa2.ty
,coalesce(sign.bb,'') as tyy
,aa2.rd ,aa2.dt ,aa2.appliedon
,aa2.approvedon ,aa2.pprec
,aa2.ppret ,aa2.cdate ,pm.pmd,tblapplic.dob
,coalesce(tblapplic.firstname ,'') as fname
,coalesce(tblapplic.middlename ,'') as mname
,coalesce(tblapplic.surname ,'') as sname
,coalesce(Upper(dcou.descr),'') as dcouu
,coalesce(aa2.visaid,0) as visaid
,coalesce(tblcountry.descr,'') as visaname
,(SELECT chr(9) || array_to_string(ARRAY(SELECT descr FROM tblcountry WHERE dcountry=travdets.dcountry ORDER BY descr),chr(9))) as visanamecombo
,coalesce(aa2.stype,0) as stypeid ,coalesce(stype.descr,'') as stype
,coalesce(aa2.sts,0) as appStatusid ,coalesce(sta.descr,'') as appStatus
,lofficesuperuser.descr as lofficesuperuser,coalesce(aa2.loffice,0) as lofficeid
,coalesce(lofficesuperuser.descr,coalesce(loffice.descr,'')) as loffice
,coalesce(pm.id,0) as pmid
,(SELECT chr(9) || array_to_string(ARRAY(SELECT descr FROM stype WHERE typee=aa2.ty and visaa=coalesce(aa2.visaid,0) ORDER BY descr),chr(9))) as sTypeCombo
,(SELECT chr(9) || array_to_string(ARRAY(SELECT descr FROM sta WHERE typee=aa2.ty ORDER BY descr),chr(9))) as appStatusCombo
,coalesce(aa2.colltype,0) as colltypeid
,coalesce(tblcolltype.descr,'') as colltype,aa2.embcolldt as embcolldt
,aa2.clcolldt as clbcolldt,travdets.leavingdt as leaving,coalesce(aa2.pptrnum,'') as ppTrackNumber
,coalesce(aa2.rt,0) as regtypeid,coalesce(tbllist1.descr,'') as regtype
,coalesce(aa2.reff,0) as referralid ,coalesce(tblreferral.descr,'') as referral
,coalesce(aa2.scaller,0) as callerid
,coalesce(namess.descr,'') as caller,coalesce(aa2.travdets,0) as travdetsid
,coalesce(curact.luser,-1) as actionluser
,aa2.iss
from aa2
LEFT JOIN tblapplic on aa2.aid=tblapplic.idnumber
left JOIN travdets on aa2.travdets=travdets.idnumber
left JOIN pm on aa2.id=pm.aa2id and pm.pagee=aa2.ty
LEFT JOIN sign ON aa2.ty = sign.aa
LEFT JOIN tblnationality dcou ON dcou.idnumber=travdets.dcountry
left join tblcountry on aa2.visaid=tblcountry.idnumber
left join aa2pre on aa2.id=aa2pre.aa2id
left join tbluser on aa2pre.addusr=tbluser.idnumber
left join loffice on tbluser.office=loffice.idnumber
left join loffice lofficesuperuser on aa2.loffice=lofficesuperuser.idnumber
left outer join stype on aa2.stype=stype.id
left outer join sta on aa2.sts=sta.idnumber and sta.typee=aa2.ty
left join tblcolltype on aa2.colltype=tblcolltype.idnumber
left join tbllist1 on aa2.rt=tbllist1.idnumber
left join tblreferral on aa2.reff=tblreferral.idnumber
left join namess on aa2.scaller=namess.idnumber
left join aa2_curr_act curact on aa2.id=curact.aa2id
where aa2.op_status=0 and aa2.ty>0
ORDER BY aa2.aid DESC, aa2.ty DESC
LIMIT 1000
该计划:
Limit (cost=2502213.29..2502215.79 rows=1000 width=555) (actual time=569132.700..569133.021 rows=1000 loops=1)
-> Sort (cost=2502213.29..2502284.78 rows=28593 width=555) (actual time=569132.699..569132.870 rows=1000 loops=1)
Sort Key: aa2.aid, aa2.ty
Sort Method: top-N heapsort Memory: 3539kB
-> Hash Left Join (cost=424537.96..2500645.57 rows=28593 width=555) (actual time=3817.372..565313.067 rows=1175709 loops=1)
Hash Cond: (aa2.id = curact.aa2id)
-> Hash Left Join (cost=424498.48..469748.41 rows=28593 width=547) (actual time=3816.435..28001.006 rows=1175709 loops=1)
Hash Cond: (aa2.scaller = namess.idnumber)
-> Hash Left Join (cost=424492.25..469350.45 rows=28593 width=532) (actual time=3816.326..27006.364 rows=1175709 loops=1)
Hash Cond: (aa2.reff = tblreferral.idnumber)
-> Hash Left Join (cost=424452.85..468924.32 rows=28593 width=506) (actual time=3815.846..25976.031 rows=1175709 loops=1)
Hash Cond: (aa2.rt = tbllist1.idnumber)
-> Nested Loop Left Join (cost=424451.44..468537.08 rows=28593 width=487) (actual time=3815.819..25098.987 rows=1175709 loops=1)
Join Filter: (aa2.colltype = tblcolltype.idnumber)"
Rows Removed by Join Filter: 8229962
-> Hash Left Join (cost=424451.44..465533.73 rows=28593 width=451) (actual time=3815.793..21647.918 rows=1175709 loops=1)
Hash Cond: ((aa2.sts = public.sta.idnumber) AND (aa2.ty = public.sta.typee))
-> Hash Left Join (cost=424443.94..464745.17 rows=28593 width=424) (actual time=3815.677..20748.314 rows=1175709 loops=1)
Hash Cond: (aa2.stype = public.stype.id)"
-> Hash Left Join (cost=424367.49..464239.83 rows=28593 width=383) (actual time=3814.669..19892.991 rows=1175709 loops=1)
Hash Cond: (aa2.loffice = lofficesuperuser.idnumber)
-> Hash Left Join (cost=424366.18..464131.28 rows=28593 width=374) (actual time=3814.651..19300.840 rows=1175709 loops=1)
Hash Cond: (tbluser.office = loffice.idnumber)
-> Hash Left Join (cost=424364.86..464002.72 rows=28593 width=373) (actual time=3814.628..18721.215 rows=1175709 loops=1)
Hash Cond: (aa2pre.addusr = tbluser.idnumber)
-> Hash Left Join (cost=424329.53..463574.24 rows=28593 width=373) (actual time=3813.991..18128.257 rows=1175709 loops=1)
Hash Cond: (aa2.id = aa2pre.aa2id)
-> Hash Left Join (cost=424309.57..463447.04 rows=28593 width=365) (actual time=3813.404..17427.883 rows=1175708 loops=1)
Hash Cond: (aa2.visaid = public.tblcountry.idnumber)
-> Hash Left Join (cost=424286.05..463120.22 rows=28593 width=342) (actual time=3813.207..16687.672 rows=1175708 loops=1)
Hash Cond: (travdets.dcountry = dcou.idnumber)
-> Hash Left Join (cost=424277.35..462718.39 rows=28593 width=330) (actual time=3813.056..15885.221 rows=1175708 loops=1)
Hash Cond: (aa2.ty = sign.aa)
-> Hash Left Join (cost=424275.90..462337.40 rows=28593 width=319) (actual time=3813.019..15036.693 rows=1175708 loops=1)
Hash Cond: (aa2.travdets = travdets.idnumber)
-> Hash Left Join (cost=408250.71..445773.33 rows=28593 width=307) (actual time=3559.650..12275.923 rows=1175708 loops=1)"
Hash Cond: (aa2.aid = tblapplic.idnumber)
-> Hash Right Join (cost=301618.16..338733.36 rows=28593 width=287) (actual time=2854.476..9334.815 rows=1175708 loops=1)
Hash Cond: ((pm.aa2id = aa2.id) AND (pm.pagee = aa2.ty))
-> Seq Scan on pm (cost=0.00..23718.76 rows=595176 width=22) (actual time=0.148..270.163 rows=583001 loops=1)
-> Hash (cost=301189.26..301189.26 rows=28593 width=275) (actual time=2854.215..2854.215 rows=1167712 loops=1)
Buckets: 4096 Batches: 4 (originally 1) Memory Usage: 65537kB
-> Seq Scan on aa2 (cost=0.00..301189.26 rows=28593 width=275) (actual time=0.010..1806.989 rows=1167712 loops=1)
Filter: ((ty > 0) AND (op_status = 0))"
Rows Removed by Filter: 98527
-> Hash (cost=98044.47..98044.47 rows=687047 width=28) (actual time=705.012..705.012 rows=447141 loops=1)
Buckets: 131072 Batches: 1 Memory Usage: 28576kB"
-> Seq Scan on tblapplic (cost=0.00..98044.47 rows=687047 width=28) (actual time=0.012..366.413 rows=447141 loops=1)
-> Hash (cost=10560.64..10560.64 rows=437164 width=20) (actual time=253.078..253.078 rows=419170 loops=1)
Buckets: 65536 Batches: 1 Memory Usage: 22917kB"
-> Seq Scan on travdets (cost=0.00..10560.64 rows=437164 width=20) (actual time=0.006..122.472 rows=419170 loops=1)
-> Hash (cost=1.20..1.20 rows=20 width=19) (actual time=0.014..0.014 rows=20 loops=1)"
Buckets: 1024 Batches: 1 Memory Usage: 2kB"
-> Seq Scan on sign (cost=0.00..1.20 rows=20 width=19) (actual time=0.004..0.009 rows=20 loops=1)
-> Hash (cost=5.53..5.53 rows=253 width=20) (actual time=0.143..0.143 rows=253 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 14kB
-> Seq Scan on tblnationality dcou (cost=0.00..5.53 rows=253 width=20) (actual time=0.007..0.064 rows=253 loops=1)
-> Hash (cost=19.90..19.90 rows=290 width=31) (actual time=0.189..0.189 rows=294 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 20kB"
-> Seq Scan on tblcountry (cost=0.00..19.90 rows=290 width=31) (actual time=0.004..0.094 rows=294 loops=1)
-> Hash (cost=17.76..17.76 rows=176 width=16) (actual time=0.576..0.576 rows=431 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 21kB
-> Seq Scan on aa2pre (cost=0.00..17.76 rows=176 width=16) (actual time=0.030..0.438 rows=494 loops=1)
-> Hash (cost=28.48..28.48 rows=548 width=16) (actual time=0.623..0.623 rows=507 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 22kB
-> Seq Scan on tbluser (cost=0.00..28.48 rows=548 width=16) (actual time=0.005..0.526 rows=507 loops=1)
-> Hash (cost=1.14..1.14 rows=14 width=17) (actual time=0.010..0.010 rows=13 loops=1)"
Buckets: 1024 Batches: 1 Memory Usage: 1kB"
-> Seq Scan on loffice (cost=0.00..1.14 rows=14 width=17) (actual time=0.003..0.005 rows=13 loops=1)"
-> Hash (cost=1.14..1.14 rows=14 width=17) (actual time=0.008..0.008 rows=13 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 1kB
-> Seq Scan on loffice lofficesuperuser (cost=0.00..1.14 rows=14 width=17) (actual time=0.002..0.005 rows=13 loops=1)
-> Hash (cost=55.09..55.09 rows=1709 width=49) (actual time=0.998..0.998 rows=1717 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 144kB
-> Seq Scan on stype (cost=0.00..55.09 rows=1709 width=49) (actual time=0.005..0.431 rows=1717 loops=1)
-> Hash (cost=5.40..5.40 rows=140 width=37) (actual time=0.100..0.100 rows=145 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 11kB
-> Seq Scan on sta (cost=0.00..5.40 rows=140 width=37) (actual time=0.004..0.046 rows=145 loops=1)
-> Materialize (cost=0.00..1.10 rows=7 width=44) (actual time=0.000..0.001 rows=7 loops=1175709)
-> Seq Scan on tblcolltype (cost=0.00..1.07 rows=7 width=44) (actual time=0.013..0.014 rows=7 loops=1)
-> Hash (cost=1.18..1.18 rows=18 width=27) (actual time=0.012..0.012 rows=18 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 2kB"
-> Seq Scan on tbllist1 (cost=0.00..1.18 rows=18 width=27) (actual time=0.004..0.007 rows=18 loops=1)
-> Hash (cost=29.18..29.18 rows=818 width=34) (actual time=0.469..0.469 rows=827 loops=1)"
Buckets: 1024 Batches: 1 Memory Usage: 57kB"
-> Seq Scan on tblreferral (cost=0.00..29.18 rows=818 width=34) (actual time=0.007..0.181 rows=827 loops=1)
-> Hash (cost=3.88..3.88 rows=188 width=23) (actual time=0.099..0.099 rows=191 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 11kB
-> Seq Scan on namess (cost=0.00..3.88 rows=188 width=23) (actual time=0.007..0.042 rows=191 loops=1)
-> Hash (cost=23.10..23.10 rows=1310 width=16) (actual time=0.000..0.000 rows=0 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 0kB
-> Seq Scan on aa2_curr_act curact (cost=0.00..23.10 rows=1310 width=16) (actual time=0.000..0.000 rows=0 loops=1)
SubPlan 2
-> Result (cost=20.66..20.67 rows=1 width=0) (actual time=0.156..0.156 rows=1 loops=1175709)
InitPlan 1 (returns $1)
-> Sort (cost=20.65..20.66 rows=3 width=23) (actual time=0.135..0.137 rows=25 loops=1175709)
Sort Key: public.tblcountry.descr
Sort Method: quicksort Memory: 28kB
-> Seq Scan on tblcountry (cost=0.00..20.62 rows=3 width=23) (actual time=0.017..0.063 rows=25 loops=1175709)
Filter: (dcountry = travdets.dcountry)
Rows Removed by Filter: 269
SubPlan 4
-> Result (cost=44.29..44.31 rows=1 width=0) (actual time=0.237..0.237 rows=1 loops=1175709)
InitPlan 3 (returns $4)
-> Sort (cost=44.29..44.29 rows=1 width=41) (actual time=0.224..0.225 rows=12 loops=1175709)
Sort Key: public.stype.descr
Sort Method: quicksort Memory: 25kB
-> Bitmap Heap Scan on stype (cost=4.93..44.28 rows=1 width=41) (actual time=0.108..0.197 rows=12 loops=1175709)
Recheck Cond: (typee = aa2.ty)
Filter: (visaa = COALESCE(aa2.visaid, 0::bigint))
Rows Removed by Filter: 629
-> Bitmap Index Scan on stypeindtypee (cost=0.00..4.93 rows=90 width=0) (actual time=0.073..0.073 rows=745 loops=1175709)
Index Cond: (typee = aa2.ty)
SubPlan 6
-> Result (cost=5.87..5.88 rows=1 width=0) (actual time=0.056..0.056 rows=1 loops=1175709)
InitPlan 5 (returns $6)
-> Sort (cost=5.85..5.87 rows=7 width=27) (actual time=0.044..0.045 rows=13 loops=1175709)
Sort Key: public.sta.descr
Sort Method: quicksort Memory: 25kB
-> Seq Scan on sta (cost=0.00..5.75 rows=7 width=27) (actual time=0.010..0.025 rows=13 loops=1175709)
Filter: (typee = aa2.ty)
Rows Removed by Filter: 132
Total runtime: 569133.611 ms
最新编辑:
对于那些没有时间解决问题的人来说,这里有一个变通方法。只需包装这个大查询,然后按下订单即可。大概是这样的:
explain ANALYSE
SELECT * FROM
(select aa2.aid as cid
,aa2.id ,aa2.ty
,coalesce(sign.bb,'') as tyy
,aa2.rd ,aa2.dt ,aa2.appliedon
,aa2.approvedon ,aa2.pprec
,aa2.ppret ,aa2.cdate ,pm.pmd,tblapplic.dob
,coalesce(tblapplic.firstname ,'') as fname
,coalesce(tblapplic.middlename ,'') as mname
,coalesce(tblapplic.surname ,'') as sname
,coalesce(Upper(dcou.descr),'') as dcouu
,coalesce(aa2.visaid,0) as visaid
,coalesce(tblcountry.descr,'') as visaname
,(SELECT chr(9) || array_to_string(ARRAY(SELECT descr FROM tblcountry WHERE dcountry=travdets.dcountry ORDER BY descr),chr(9))) as visanamecombo
,coalesce(aa2.stype,0) as stypeid ,coalesce(stype.descr,'') as stype
,coalesce(aa2.sts,0) as appStatusid ,coalesce(sta.descr,'') as appStatus
,lofficesuperuser.descr as lofficesuperuser,coalesce(aa2.loffice,0) as lofficeid
,coalesce(lofficesuperuser.descr,coalesce(loffice.descr,'')) as loffice
,coalesce(pm.id,0) as pmid
,(SELECT chr(9) || array_to_string(ARRAY(SELECT descr FROM stype WHERE typee=aa2.ty and visaa=coalesce(aa2.visaid,0) ORDER BY descr),chr(9))) as sTypeCombo
,(SELECT chr(9) || array_to_string(ARRAY(SELECT descr FROM sta WHERE typee=aa2.ty ORDER BY descr),chr(9))) as appStatusCombo
,coalesce(aa2.colltype,0) as colltypeid
,coalesce(tblcolltype.descr,'') as colltype,aa2.embcolldt as embcolldt
,aa2.clcolldt as clbcolldt,travdets.leavingdt as leaving,coalesce(aa2.pptrnum,'') as ppTrackNumber
,coalesce(aa2.rt,0) as regtypeid,coalesce(tbllist1.descr,'') as regtype
,coalesce(aa2.reff,0) as referralid ,coalesce(tblreferral.descr,'') as referral
,coalesce(aa2.scaller,0) as callerid
,coalesce(namess.descr,'') as caller,coalesce(aa2.travdets,0) as travdetsid
,coalesce(curact.luser,-1) as actionluser
,aa2.iss
from aa2
LEFT JOIN tblapplic on aa2.aid=tblapplic.idnumber
left JOIN travdets on aa2.travdets=travdets.idnumber
left JOIN pm on aa2.id=pm.aa2id and pm.pagee=aa2.ty
LEFT JOIN sign ON aa2.ty = sign.aa
LEFT JOIN tblnationality dcou ON dcou.idnumber=travdets.dcountry
left join tblcountry on aa2.visaid=tblcountry.idnumber
left join aa2pre on aa2.id=aa2pre.aa2id
left join tbluser on aa2pre.addusr=tbluser.idnumber
left join loffice on tbluser.office=loffice.idnumber
left join loffice lofficesuperuser on aa2.loffice=lofficesuperuser.idnumber
left outer join stype on aa2.stype=stype.id
left outer join sta on aa2.sts=sta.idnumber and sta.typee=aa2.ty
left join tblcolltype on aa2.colltype=tblcolltype.idnumber
left join tbllist1 on aa2.rt=tbllist1.idnumber
left join tblreferral on aa2.reff=tblreferral.idnumber
left join namess on aa2.scaller=namess.idnumber
left join aa2_curr_act curact on aa2.id=curact.aa2id
where aa2.op_status=0 and aa2.ty>0)
ORDER BY aa2.aid DESC, aa2.ty DESC
LIMIT 1000
这背后的原因是您的查询没有使用该表的任何索引 对于需要扫描大部分表的查询,显式排序可能比使用索引更快,因为它需要更少的磁盘I/O,因为它遵循顺序访问模式。当只需要提取几行时,索引更有用 如果一个
ORDERY BY
可以使用索引,您将在EXPLAIN
中看到如下内容:
Index Scan using test_idx on test_tbl (cost=0.15..24.28 rows=285 width=27)
F.ex。在我的(较小的)表中,如果使用了LIMIT 210
,则使用此索引,但如果我提供了LIMIT 211
,则使用整个表扫描。(但是,您的表可能会有所不同,在决定是否使用索引时,查询计划器可以考虑行数和行宽。)
tbl\u name
在受影响的表上,这将刷新统计信息work\u mem
设置得更高一点,这可以使引擎将其哈希表保留在核心中join\u collapse\u limit
(默认设置为8)。尝试将其设置得更高一点,这可以使乐观者看到超出其当前范围的东西effective\u cach\u size
设置为至少一半物理内存,并将random\u page\u cost
设置为低于默认值4的值(尝试1.5)[所有配置设置都可以在current.sql中设置,方法是发出
set parameter\u name=value;
]更新统计信息是永久性的,但不会造成任何伤害。对于那些没有时间解决问题的人,这里有一个解决方法。只需包装大查询,然后按进行排序。查询计划显示排序在~0.2毫秒内完成。排序前的查询部分为-557395.213毫秒。您可以尝试在不进行排序的情况下执行相同的查询,并检查其计划。限制(成本=44.94..93484.76行=1000宽度=555)(实际时间=1.283..391.665行=1000圈=1)->嵌套循环左连接(成本=44.94..2671769.72行=28593宽度=555)(实际时间=1.282..391.253行=1000个循环=1)…-->执行时间:562ms;总时间:625毫秒这是一个非常不同的执行计划。它不需要读取所有数据进行排序。发布完整的查询和查询计划。我已经编辑了原始帖子。我如何强制使用索引扫描来比较两次执行?您只能全局执行,并且不适用于正在进行的生产使用:最好是重新构造查询。@pozs您不仅可以全局禁用seq扫描,还可以在会话/事务级别禁用seq扫描。@IgorRomanchenko是的,我的意思是不能在单个表/查询中强制索引(即禁用seq扫描)。谢谢您的回复。我已经对所有表格进行了真空分析;work\u mem
设置为64MB;join\u collapse\u limit
设置为20;有效缓存大小为14GB;随机页面成本为1.5;只有join\u collapse\u limit
会影响执行时间,将执行时间从9分钟以上减少到1分钟。但是,如果我删除ORDER BY语句,则查询执行时间少于一秒。请将(秒)结果计划添加到问题中。无法添加,因为我超过了允许的最大符号数:(大家好,我不明白问题出在哪里,但我没时间了,没有更多的时间来解决它。相反,我做了一个变通办法。我将编辑原始帖子