Sql 选择前N行，其中文本字段的长度总和达到某个限制

我有一张这样的桌子：

CREATE TABLE cache (
  id BIGSERIAL PRIMARY KEY,
  source char(2) NOT NULL,
  target char(2) NOT NULL,
  q TEXT NOT NULL,
  result TEXT,
  profile TEXT NOT NULL DEFAULT '',
  created TIMESTAMP NOT NULL DEFAULT now(),
  api_engine text NOT NULL,
  encoded TEXT NOT NULL
);

我想通过编码字段列表（可能通过…窗口？） 比如：

SELECT id, string_agg(encoded, '&q=') FROM cache

因此，我将有相应ID的列表和一个编码的串接字段：

”&q=encoded1&q=encoded2&q=encoded3'

。。。总长度不超过某些限制（例如不超过2000个字符）

第二个条件，我想转到下一个窗口，当其中一个字段：source、target或profile发生更改时

如果可能的话，使用SQL选择FOR循环

我知道如何使用plpgsql/plpython/plperl实现这一点，但我想优化这个请求

FOR rec IN
  SELECT array_agg(id) AS ids, string_agg(encoded, '&q=') AS url FROM cache
  WHERE result IS NULL
  ORDER BY source, target
LOOP
  -- here I call curl with that *url*

示例数据：

INSERT INTO cache (id, source, target, q, result, profile, api_engine, encoded) VALUES
   (1, 'ru', 'en', 'Длинная фраза по-русски'            , NULL, '', 'google', '%D0%94%D0%BB%D0%B8%D0%BD%D0%BD%D0%B0%D1%8F+%D1%84%D1%80%D0%B0%D0%B7%D0%B0+%D0%BF%D0%BE-%D1%80%D1%83%D1%81%D1%81%D0%BA%D0%B8')
 , (2, 'ru', 'es', 'Ещё одна непонятная фраза по-русски', NULL, '', 'google', '%D0%95%D1%89%D1%91+%D0%BE%D0%B4%D0%BD%D0%B0+%D0%BD%D0%B5%D0%BF%D0%BE%D0%BD%D1%8F%D1%82%D0%BD%D0%B0%D1%8F+%D1%84%D1%80%D0%B0%D0%B7%D0%B0+%D0%BF%D0%BE-%D1%80%D1%83%D1%81%D1%81%D0%BA%D0%B8')
-- etc...

以此类推，像这样的行有100500行。字段source和target可以是不同的语言代码，它们重复，所以我可能需要按source、target、profile进行分组

我想选择前N行，其中字段的串联使用一些分隔符编码，如

因此，此连接字符串的长度不超过（2000）个字符。因此，我将在url中包含该字符串以及这些行的所有ID（当然，顺序相同）

---

然后我想用相同的条件选择接下来的N行，依此类推。

您可以使用智能递归CTE执行此操作：

WITH RECURSIVE c AS ( -- 1st CTE is not recursive
   SELECT dense_rank()  OVER (ORDER BY     source, target, profile)             AS rnk
        , row_number()  OVER (PARTITION BY source, target, profile ORDER BY id) AS rn
        , lead(encoded) OVER (PARTITION BY source, target, profile ORDER BY id) AS next_enc
        , id, encoded
   FROM   cache
   )

 , rcte AS (  -- "recursion" starts here
   SELECT rnk, rn, ARRAY[id] AS ids, encoded AS url
        , CASE WHEN length(concat_ws('&q=', encoded || next_enc)) > 2000  -- max len
                 OR next_enc IS NULL  -- last in partition
               THEN TRUE END AS print
   FROM   c
   WHERE  rn = 1

   UNION ALL
   SELECT c.rnk, c.rn
        , CASE WHEN r.print THEN ARRAY[id] ELSE r.ids || c.id                      END AS ids
        , CASE WHEN r.print THEN c.encoded ELSE concat_ws('&q=', r.url, c.encoded) END AS url
        , CASE WHEN length(
             CASE WHEN r.print THEN concat_ws('&q=', c.encoded, c.next_enc)
                  ELSE concat_ws('&q=', r.url, c.encoded, c.next_enc) END) > 2000  -- max len
                 OR c.next_enc IS NULL  -- last in partition
               THEN TRUE END AS print
   FROM   rcte r
   JOIN        c USING (rnk)
   WHERE  c.rn = r.rn + 1
   )
SELECT ids, url
FROM   rcte
WHERE  print
ORDER  BY rnk, rn;

关于rCTE，包括非递归CTE：

WITH RECURSIVE c AS ( -- 1st CTE is not recursive
   SELECT dense_rank()  OVER (ORDER BY     source, target, profile)             AS rnk
        , row_number()  OVER (PARTITION BY source, target, profile ORDER BY id) AS rn
        , lead(encoded) OVER (PARTITION BY source, target, profile ORDER BY id) AS next_enc
        , id, encoded
   FROM   cache
   )

 , rcte AS (  -- "recursion" starts here
   SELECT rnk, rn, ARRAY[id] AS ids, encoded AS url
        , CASE WHEN length(concat_ws('&q=', encoded || next_enc)) > 2000  -- max len
                 OR next_enc IS NULL  -- last in partition
               THEN TRUE END AS print
   FROM   c
   WHERE  rn = 1

   UNION ALL
   SELECT c.rnk, c.rn
        , CASE WHEN r.print THEN ARRAY[id] ELSE r.ids || c.id                      END AS ids
        , CASE WHEN r.print THEN c.encoded ELSE concat_ws('&q=', r.url, c.encoded) END AS url
        , CASE WHEN length(
             CASE WHEN r.print THEN concat_ws('&q=', c.encoded, c.next_enc)
                  ELSE concat_ws('&q=', r.url, c.encoded, c.next_enc) END) > 2000  -- max len
                 OR c.next_enc IS NULL  -- last in partition
               THEN TRUE END AS print
   FROM   rcte r
   JOIN        c USING (rnk)
   WHERE  c.rn = r.rn + 1
   )
SELECT ids, url
FROM   rcte
WHERE  print
ORDER  BY rnk, rn;

但这可能是plpgsql函数中的循环实际上更快的极少数情况之一

有关更多说明，请参阅此相关答案：

---

您可以编辑您的问题并提供示例数据和所需结果吗？如果长度超过2000个字符，会发生什么情况？如果长度超过配额，我会将结果推入函数，该函数将从http API发送/接收缓慢的答案，然后从表的下一部分开始。很抱歉，我的定义不准确。确保请求将进入IN SELECT循环。

encoded

未定义非空？前导的

http://google.translation-api.com/api/v2&q=

来自哪里？它是否计入2000个字符的限制？你对博士后的看法是什么？谢谢你，欧文，我从没听说过CTE。它看起来比普通循环复杂得多。正是你的请求返回了副本。ID，url:{1,2,3,4,18,19,21,22,23,25,37}，%%%此处为长字符串'；{1,2,3,4,18,19,21,22,23,25,37,38}，“%%%以前的字符串+一些数据”；{1,2,3,4,18,19,21,22,23,25,37,38,39}，“%等”@迪米特里：对不起，我很忙，打印后忘了重新启动聚合。现在修好了。无论如何，这只是一个概念的证明。我几乎可以肯定，对于这种特殊情况，在表格上循环会更快、更简单。