Postgresql 是否要更新具有GROUP BY的子查询中的WHERE值，以便不存在争用条件问题？

也许这是我的天真，也许是我的偏执，但我想我正在寻找一个解决种族状况问题的方法，这个问题似乎很普遍，会有大量的解决方案，而我现在已经找到了一个。。。但我没有

最简单的情况是，我有一个进程，它应该能够抓取任何记录，其中有多个特定类型的记录。我想让系统/进程（es）线程-/多处理-/可重入-/当今流行语安全；如果同样的过程开始了，并引入了一个竞赛条件，试图抓住感兴趣的行，我希望有明确的赢家/输家：一个成功，另一个错误；事实上，我更喜欢第二次的无缝、无声、优雅的“失败”，因为它只会看不到那些会被第一次抓住的东西

这就是我的困境

我的问题如下：

   UPDATE my_table
      SET processing_by = our_id_info -- unique to this worker
    WHERE trans_nbr IN (
                          SELECT trans_nbr
                            FROM my_table
                        GROUP BY trans_nbr
                          HAVING COUNT(trans_nbr) > 1
                           LIMIT our_limit_to_have_single_process_grab
                       )
RETURNING row_id

   UPDATE my_table
      SET processing_by = our_id_info -- unique to this worker
    WHERE trans_nbr IN (
                        -- This query MAY pull ones we don't want to mess with (already "grabbed")
                          SELECT trans_nbr
                            FROM my_table
                        GROUP BY trans_nbr
                          HAVING COUNT(*) > 1
                           LIMIT our_limit_to_have_single_process_grab
                             AND processing_by IS NULL -- only "ungrabbed" ones (at this point)
                       )
      AND processing_by IS NULL -- But THIS will drop out any "bogus" ones that changed between subquery and here
RETURNING row_id

   UPDATE my_table
      SET processing_by = our_id_info -- unique to this worker
    WHERE trans_nbr IN (
            -- This query MAY pull ones we don't want to mess with (already "grabbed")
              SELECT trans_nbr
                FROM my_table AS inner_my_table_1
            GROUP BY trans_nbr
              HAVING Count(*) > 1
                 AND Count(*) in ( -- For MY query, since I'm grouping-by, I want "all or none" of trans_nbr rows
                       SELECT Count(*)
                         FROM my_table AS inner_my_table_2
                        WHERE inner_my_table_2.trans_nbr = inner_my_table_1.trans_nbr
                          AND pg_try_advisory_xact_lock(id) -- INT that will uniquely ID this row
                                 )
/* Note also that this will still lock all non-locked rows with this
   trans_nbr, even though we won't use them unless we can grab ALL of the
   rows with same trans_nbr... the rest of our query should be made
   quick-enough to accept this reality and not tie up the server unduly.

   See linked info for more-simple queries not doing group-by's.
*/
               LIMIT our_limit_to_have_single_process_grab
                 AND processing_by IS NULL -- only "ungrabbed" ones (at this point)
                       )
      AND processing_by IS NULL -- But THIS will drop out any "bogus" ones that changed between subquery and here
RETURNING row_id

我的想法是：我认为没有锁，因此无法保证子查询和外部更新之间的“状态”。那么，如何确保这个过程中得到的任何候选人，我们抓住了，而他们在同一时间没有被另一个过程抓住

我曾考虑过在子查询的末尾添加一个“ON my_table”，但这行不通；不能有这个和“分组依据”（这是计算trans_nbr数量所必需的）。（由于这将迫使任何RAN在更新之前被阻止，因此这将是一个首选解决方案，因为这样可以避免由竞争条件引起的错误[两个进程捕获同一行{s}]并允许那些其他进程在不知不觉中愉快地运行，只得到不再包含第一个进程捕获的行。唉。）

我曾经考虑过锁定表，但是（至少在Postgres中）表锁只有在提交后才被释放；出于测试目的，我不想提交，因此在测试期间（是的，在测试数据库上进行测试之后，在实时数据库上进行预实时测试），不需要这样做。（此外，即使是实时的，如果有足够的用户/进程，这也会给性能带来不可接受的影响。）

我曾经考虑过让更新依赖于子查询的processing_by值，但是，同样，这是行不通的：如果在子查询中，将打破GROUP by/HAVING条件（因为现在会有trans_nbr/processing_的子组被计数，这不是我想要的）

我期待着在正确的方向上有一些尖锐的观点嘲笑我提出这样一个显而易见的问题，但这对我来说并不明显（显然；o），我向你保证，我已经研究了几个小时了

非常感谢您的提示，更不用说解决方案了

---

更新：非常感谢

我想起了关于“”的那句老话！：>

这里是查询的修改版本，考虑了这个建议，并添加了另一个“双重检查”。这应该是一个

   UPDATE my_table
      SET processing_by = our_id_info -- unique to this worker
    WHERE trans_nbr IN (
                SELECT trans_nbr
                  FROM my_table
                 WHERE trans_nbr IN (
                           SELECT trans_nbr
                             FROM my_table
                         GROUP BY trans_nbr
                           HAVING COUNT(*) > 1 -- Thanks for the suggestion, Flimzy
                            LIMIT our_limit_to_have_single_process_grab
                                    )
                   AND processing_by IS NULL
                       /* Or some other logic that says "not currently being
                          processed".  This way, we ALSO verify we're not
                          grabbing one that might have been UPDATEd/grabbed
                          during our sub-SELECT, while it was being
                          blocked/waiting.

                          This COULD go in our UPDATE/top-level, but unnecessary
                          rows could be locked by this lower-level in that case.
                       */
            FOR UPDATE /* Will block/wait for rows this finds to be unlocked by
                          any prior transaction that had a lock on them.

                          NOTE: Which _could_ allow the prior trans to change
                                our desired rows in the mean time, thus the
                                secondary WHERE clause.
                       */
                       )
RETURNING row_id

我希望博士后有一个类似的功能。特别是对于需要在不阻塞其他处理的情况下处理的基本上是原子行的队列。：-）

目前，您可能会添加一个选项，以避免被任何其他事务阻塞，但是请记住，它只会返回一个错误-您必须继续尝试您的查询，直到它成功（或放弃）。如果没有NOWAIT，查询将阻塞，直到其他事务释放其锁，或者查询超时

---

更新2:因此，在重新阅读并思考这篇文章之后，再次“Forrest for the Trees”一刻。我可以这样做：

   UPDATE my_table
      SET processing_by = our_id_info -- unique to this worker
    WHERE trans_nbr IN (
                          SELECT trans_nbr
                            FROM my_table
                        GROUP BY trans_nbr
                          HAVING COUNT(trans_nbr) > 1
                           LIMIT our_limit_to_have_single_process_grab
                       )
RETURNING row_id

   UPDATE my_table
      SET processing_by = our_id_info -- unique to this worker
    WHERE trans_nbr IN (
                        -- This query MAY pull ones we don't want to mess with (already "grabbed")
                          SELECT trans_nbr
                            FROM my_table
                        GROUP BY trans_nbr
                          HAVING COUNT(*) > 1
                           LIMIT our_limit_to_have_single_process_grab
                             AND processing_by IS NULL -- only "ungrabbed" ones (at this point)
                       )
      AND processing_by IS NULL -- But THIS will drop out any "bogus" ones that changed between subquery and here
RETURNING row_id

   UPDATE my_table
      SET processing_by = our_id_info -- unique to this worker
    WHERE trans_nbr IN (
            -- This query MAY pull ones we don't want to mess with (already "grabbed")
              SELECT trans_nbr
                FROM my_table AS inner_my_table_1
            GROUP BY trans_nbr
              HAVING Count(*) > 1
                 AND Count(*) in ( -- For MY query, since I'm grouping-by, I want "all or none" of trans_nbr rows
                       SELECT Count(*)
                         FROM my_table AS inner_my_table_2
                        WHERE inner_my_table_2.trans_nbr = inner_my_table_1.trans_nbr
                          AND pg_try_advisory_xact_lock(id) -- INT that will uniquely ID this row
                                 )
/* Note also that this will still lock all non-locked rows with this
   trans_nbr, even though we won't use them unless we can grab ALL of the
   rows with same trans_nbr... the rest of our query should be made
   quick-enough to accept this reality and not tie up the server unduly.

   See linked info for more-simple queries not doing group-by's.
*/
               LIMIT our_limit_to_have_single_process_grab
                 AND processing_by IS NULL -- only "ungrabbed" ones (at this point)
                       )
      AND processing_by IS NULL -- But THIS will drop out any "bogus" ones that changed between subquery and here
RETURNING row_id

提交事务以释放我们的锁，以及Bob的叔叔

尽管如此，SKIP LOCKED仍然非常酷

警告：如果要让工人拉有限（如LIMIT 1）数量的行和/或项目，必须按特定顺序抓取（例如：FIFO，按顺序和/或按函数（如Min（id））抓取），可能会出现工人饿死的情况：工人等待并等待，当他们等待解锁行时，结果表明，它们都不符合最终标准。有很多方法可以解决这个问题，比如让工人通过偏移跳转，但大多数方法要么复杂，要么缓慢。（通常两者都有。奖金！）

我的功能性要求返回多行，或者没有一行是可以的-暂时无事可做；睡一会儿再检查一下，所以这对我来说不是问题。也许是给你的。如果是这样，你会想考虑……/P> 非阻塞版本：我发现了一个解决这一问题的方法，它将我介绍给了Pg。（信息也相当丰富。）

因此，我自己问题的非阻塞解决方案应该如下所示：

   UPDATE my_table
      SET processing_by = our_id_info -- unique to this worker
    WHERE trans_nbr IN (
                          SELECT trans_nbr
                            FROM my_table
                        GROUP BY trans_nbr
                          HAVING COUNT(trans_nbr) > 1
                           LIMIT our_limit_to_have_single_process_grab
                       )
RETURNING row_id

   UPDATE my_table
      SET processing_by = our_id_info -- unique to this worker
    WHERE trans_nbr IN (
                        -- This query MAY pull ones we don't want to mess with (already "grabbed")
                          SELECT trans_nbr
                            FROM my_table
                        GROUP BY trans_nbr
                          HAVING COUNT(*) > 1
                           LIMIT our_limit_to_have_single_process_grab
                             AND processing_by IS NULL -- only "ungrabbed" ones (at this point)
                       )
      AND processing_by IS NULL -- But THIS will drop out any "bogus" ones that changed between subquery and here
RETURNING row_id

   UPDATE my_table
      SET processing_by = our_id_info -- unique to this worker
    WHERE trans_nbr IN (
            -- This query MAY pull ones we don't want to mess with (already "grabbed")
              SELECT trans_nbr
                FROM my_table AS inner_my_table_1
            GROUP BY trans_nbr
              HAVING Count(*) > 1
                 AND Count(*) in ( -- For MY query, since I'm grouping-by, I want "all or none" of trans_nbr rows
                       SELECT Count(*)
                         FROM my_table AS inner_my_table_2
                        WHERE inner_my_table_2.trans_nbr = inner_my_table_1.trans_nbr
                          AND pg_try_advisory_xact_lock(id) -- INT that will uniquely ID this row
                                 )
/* Note also that this will still lock all non-locked rows with this
   trans_nbr, even though we won't use them unless we can grab ALL of the
   rows with same trans_nbr... the rest of our query should be made
   quick-enough to accept this reality and not tie up the server unduly.

   See linked info for more-simple queries not doing group-by's.
*/
               LIMIT our_limit_to_have_single_process_grab
                 AND processing_by IS NULL -- only "ungrabbed" ones (at this point)
                       )
      AND processing_by IS NULL -- But THIS will drop out any "bogus" ones that changed between subquery and here
RETURNING row_id

注意事项：

• 这取决于应用程序是否使用/尊重建议锁，因此这不是pancea，也不是安慰剂。同样，由于这一点，SKIP LOCKED非常方便
• ，因为V8.2不自动解锁，（因此）可能（必须）被显式解锁
• ，因为V9.1，在事务结束时自动解锁，可能不会显式解锁
• 我还没有测试过这个！我将在我有

---

为锁定增加一个子查询层怎么样

   UPDATE my_table
      SET processing_by = our_id_info -- unique to this instance
    WHERE trans_nbr IN (
                    SELECT trans_nbr
                      FROM my_table
                     WHERE trans_nbr IN (
                                 SELECT trans_nbr
                                   FROM my_table
                               GROUP BY trans_nbr
                                 HAVING COUNT(trans_nbr) > 1
                                  LIMIT our_limit_to_have_single_process_grab
                                 )
                        FOR UPDATE
                       )
RETURNING row_id

---

为锁定增加一个子查询层怎么样

   UPDATE my_table
      SET processing_by = our_id_info -- unique to this instance
    WHERE trans_nbr IN (
                    SELECT trans_nbr
                      FROM my_table
                     WHERE trans_nbr IN (
                                 SELECT trans_nbr
                                   FROM my_table
                               GROUP BY trans_nbr
                                 HAVING COUNT(trans_nbr) > 1
                                  LIMIT our_limit_to_have_single_process_grab
                                 )
                        FOR UPDATE
                       )
RETURNING row_id

---

你有没有想过在两个SQL会话中测试这个问题？小结：除非trans\u nbr有时为NULL，否则使用

COUNT（*）

和

COUNT（trans\u nbr）

可能会得到更好的结果，因为我相信前者可以得到更好的优化。@MikeSherrill'Catcall'：是的，我已经考虑过了。开始处理一些测试数据/查询，但被调用执行其他任务。（我现在再次讨论这个问题，因为有人提出了一个解决方案。从逻辑上讲，我提到的问题是“理论上可能的”，不管这个问题是否被看到。2.即使通过测试，这些类型的concurr