python sqlite3中的递归（级联）选择

我有一个sqlite表，有3列，分别命名为ID（integer）、N（integer）和V（real）。该对（ID，N）是唯一的

使用python模块sqlite3，我想对表单执行递归选择

select ID from TABLE where N = 0 and V between ? and ? and ID in 
    (select ID from TABLE where N = 7 and V between ? and ? and ID in
        (select ID from TABLE where N = 8 and V between ? and ? and ID in  
            (...)
        )
    )

我得到以下错误，可能是因为超过了最大递归深度（？）。我需要大约20到50个复发级别

sqlite3.OperationalError: parser stack overflow

我也试着加入一些子选择，比如

select ID from
         (select ID from TABLE where N = 0 and V between ? and ?) 
    join (select ID from TABLE where N = 7 and V between ? and ?) using (ID)
    join (select ID from TABLE where N = 8 and V between ? and ?) using (ID)
    join ...

但这一方法速度惊人地慢，即使只有很少的子选择

是否有更好的方法执行相同的选择？
注：该表在（N，V）上编制索引

下面是一个示例，演示了选择是如何工作的

ID  N   V  
0   0   0,1  
0   1   0,2  
0   2   0,3  
1   0   0,5  
1   1   0,6  
1   2   0,7  
2   0   0,8  
2   1   0,9  
2   2   1,2

步骤0

select ID from TABLE where N = 0 and V between 0 and 0,6

ID位于（0,1）
第一步

ID仍在（0,1）
步骤2

---

ID为1

打开递归，按相反顺序执行，然后用Python执行。为此，我创建了一个由100条记录组成的表，每条记录的Id介于0和99之间，N=3和V=5。我任意选择了整个记录集合作为最里面的

您需要想象有一个索引了N和V的值列表，以便为最后一次SQL选择选择列表开头的值。循环所做的只是将内部SELECT生成的ID列表作为IN子句的一部分提供给下一个SELECT

如果没有任何索引，这一切都将在奥根布里克完成

>>> import sqlite3
>>> conn = sqlite3.connect('recur.db')
>>> c = conn.cursor()
>>> previous_ids = str(tuple(range(0,100)))
>>> for it in range(50):
...     rows = c.execute('''SELECT ID FROM the_table WHERE N=3 AND V BETWEEN 2 AND 7 AND ID IN %s''' % previous_ids)
...     previous_ids = str(tuple([int(_[0]) for _ in rows.fetchall()]))
...     
>>> previous_ids
'(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99)'

编辑：这避免了使用长字符串，比augenblick需要更长的时间。这与使用表实现的想法基本相同

>>> import sqlite3
>>> conn = sqlite3.connect('recur.db')
>>> c = conn.cursor()
>>> N_V = [
... (0, (0,6)),
... (0, (0, 1)),
... (1, (0, 2)),
... (2, (0, 3)),
... (0, (0, 5)),
... (1, (0, 6)),
... (2, (0, 7)),
... (0, (0, 8)),
... (1, (0, 9)),
... (2, (1, 2))
... ]
>>> r = c.execute('''CREATE TABLE essentials AS SELECT ID, N, V FROM the_table WHERE N=0 AND V BETWEEN 0 AND 6''')
>>> for n_v in N_V[1:]:
...     r = c.execute('''CREATE TABLE next AS SELECT * FROM essentials WHERE essentials.ID IN (SELECT ID FROM the_table WHERE N=%s AND V BETWEEN %s AND %s)''' % (n_v[0], n_v[1][0], n_v[1][1]))
...     r = c.execute('''DROP TABLE essentials''')
...     r = c.execute('''ALTER TABLE next RENAME TO essentials''')
... 

索引三元组（ID，N，V）而不仅仅是（N，V）双元组使得连接方法足够快，可以考虑

create index I on TABLE(ID, N, V)  

然后

select ID from
         (select ID from TABLE where N = 0 and V between ? and ?) 
    join (select ID from TABLE where N = 7 and V between ? and ?) using (ID)
    join (select ID from TABLE where N = 8 and V between ? and ?) using (ID)
    join ...

---

此查询只需要子查询的（N，V）索引。ID上的单独索引可能有助于外部查询：

select ID from t
where ID in (select ID from TABLE where N = 0 and V between ? and ?)
  and ID in (select ID from TABLE where N = 7 and V between ? and ?)
  and ID in (select ID from TABLE where N = 8 and V between ? and ?)
  ...

---

V

边界在每个步骤中是否相同？

N

值来自何处？是否需要所有步骤的ID？否每个步骤中的V值都不同。N值是任意的，仅用于示例。我们的想法只是一步一步地优化选择，通常有多少个步骤？您想要所有步骤的ID吗？通常有20到50个步骤。是的，我只想得到所有步骤都能接受的ID。选择2或3个步骤效果很好。谢谢你的回答，这确实是一个很好的方法。您认为这种方法可以处理大型表吗？因为（1）我认为如果使用数字列表，IN语句的长度是有限的，（2）这种方法需要加载大量数据（至少在第一次迭代中是如此），这会降低选择速度。我没有考虑过限制数字列表长度的解决方法；我相信你是对的。“此方法意味着加载[a]大量数据”是什么意思？我的意思是，“以前的_id”列表在第一步中可能有很多项（在我的情况下是数百万项），然后fetchall命令需要时间。我怀疑我用这个示例误导了您。

select ID from
         (select ID from TABLE where N = 0 and V between ? and ?) 
    join (select ID from TABLE where N = 7 and V between ? and ?) using (ID)
    join (select ID from TABLE where N = 8 and V between ? and ?) using (ID)
    join ...

select ID from t
where ID in (select ID from TABLE where N = 0 and V between ? and ?)
  and ID in (select ID from TABLE where N = 7 and V between ? and ?)
  and ID in (select ID from TABLE where N = 8 and V between ? and ?)
  ...