Orm s:1次查询以获取所有主记录，N次查询（每个主记录一次）以获取每个主记录的所有详细信息_Orm_Select N Plus 1

Orm s:1次查询以获取所有主记录，N次查询（每个主记录一次）以获取每个主记录的所有详细信息

orm

Orm s:1次查询以获取所有主记录，N次查询（每个主记录一次）以获取每个主记录的所有详细信息,orm,select-n-plus-1,Orm,Select N Plus 1,更多数据库查询调用→ 更多延迟时间→ 降低了应用程序/数据库性能但是，ORM有避免此问题的选项，主要是使用联接。正如其他人更优雅地指出的那样，问题在于您要么拥有一个单列的笛卡尔乘积，要么正在进行N+1选择。可能是巨大的结果集，也可能是数据库的聊天室我很惊讶没有提到这一点，但这是我如何绕过这个问题的我创建一个半临时的ids表这并不适用于所有情况（可能甚至不是大多数），但如果您有很多子对象，笛卡尔积将失控（即大量OneToMany列，结果的数量将是列的乘法），并且这更像是一个批处理作业，那么它

更多数据库查询调用→ 更多延迟时间→ 降低了应用程序/数据库性能

但是，ORM有避免此问题的选项，主要是使用联接。

正如其他人更优雅地指出的那样，问题在于您要么拥有一个单列的笛卡尔乘积，要么正在进行N+1选择。可能是巨大的结果集，也可能是数据库的聊天室

我很惊讶没有提到这一点，但这是我如何绕过这个问题的我创建一个半临时的ids表

这并不适用于所有情况（可能甚至不是大多数），但如果您有很多子对象，笛卡尔积将失控（即大量

OneToMany

列，结果的数量将是列的乘法），并且这更像是一个批处理作业，那么它的效果会特别好

首先，将父对象ID作为批插入ids表中。这个批处理id是我们在应用程序中生成并保留的

INSERT INTO temp_ids 
    (product_id, batch_id)
    (SELECT p.product_id, ? 
    FROM product p ORDER BY p.product_id
    LIMIT ? OFFSET ?);

现在，对于每个

OneToMany

列，您只需在ids表

内部联接上执行SELECT
，并使用WHERE batch\u id=
（反之亦然）连接子表。您只需要确保按id列排序，因为这将使合并结果列变得更容易（否则，您需要为整个结果集提供一个HashMap/表，这可能没有那么糟糕）
然后定期清理ids表
如果用户选择100个左右不同的项目进行某种批量处理，这种方法也特别有效。将100个不同的ID放入临时表中
现在，您所做的查询的数量是由一个多个列的数量决定的。
一个百万富翁拥有N辆汽车。你想得到所有的轮子
一（1）个查询加载所有车辆，但对于每（N）个车辆，提交一个单独的车轮加载查询
费用：
假设索引适合ram
1+N查询解析和规划+索引搜索和1+N+（N*4）板访问加载有效负载
假设索引不适合ram
在最坏情况下，加载索引的1+N板访问的额外成本
总结
瓶颈是板访问（硬盘上每秒约70次随机访问）
对于有效载荷，急连接选择也将访问板1+N+（N*4）次。
因此，如果索引适合ram-没问题，速度足够快，因为只涉及ram操作。
以Matt Solnit为例，假设您将汽车和车轮之间的关联定义为惰性，并且需要一些车轮字段。这意味着在第一次选择之后，hibernate将为每辆车执行“从车轮选择*，其中car_id=：id”
这使得每N辆车的第一个选择和更多的1个选择，这就是为什么它被称为N+1问题
要避免这种情况，请将关联获取设置为“急切”，以便hibernate使用联接加载数据
但请注意，如果您多次不访问相关的控制盘，最好让它保持惰性或使用条件更改获取类型。
我不能直接评论其他答案，因为我没有足够的声誉。但值得注意的是，这个问题的出现主要是因为，从历史上看，很多dbms在处理连接时都非常糟糕（MySQL是一个特别值得注意的例子）。因此，n+1通常比join快得多。还有一些方法可以改进n+1，但仍然不需要连接，这就是原始问题所涉及的
然而，在连接方面，MySQL现在比以前好多了。当我第一次学习MySQL时，我经常使用连接。然后我发现它们有多慢，并在代码中切换到n+1。但是，最近，我又回到了连接，因为MySQL现在在处理连接方面比我第一次使用它时要好得多
现在，从性能方面来说，在一组索引正确的表上进行简单的连接很少会有问题。如果它确实给性能带来了影响，那么使用索引提示通常可以解决这些问题
MySQL开发团队的一位成员在这里讨论了这一点：

因此，总结如下：如果您过去一直因为MySQL糟糕的性能而避免加入，那么请在最新版本上再试一次。您可能会感到惊喜。
发出一个返回100个结果的查询要比发出100个每个返回1个结果的查询快得多。

只需向测试类添加一个特殊的JUnit规则，并在测试方法上放置带有预期查询数的注释：
@Rule
public final QueryCounter queryCounter = new QueryCounter();

@Expectation(atMost = 3)
@Test
public void testInvokingDatabase() {
    // your JDBC or JPA code
}

什么是N+1查询问题
当数据访问框架执行N条附加SQL语句以获取在执行主SQL查询时可能检索到的相同数据时，就会出现N+1查询问题
N的值越大，执行的查询越多，对性能的影响就越大。而且，与可以帮助您查找慢速运行查询的慢速查询日志不同，N+1问题不会被发现，因为每个附加查询的运行速度都足够快，不会触发慢速查询日志
问题在于执行大量额外的查询，总的来说，这些查询需要足够的时间来降低响应时间
让我们考虑下面的POST和POST注释数据库表，它们形成一对多表关系：

我们将创建以下4个post
行：
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 1', 1)
 
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 2', 2)
 
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 3', 3)
 
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 4', 4)

此外，我们还将创建4个post_comment
子记录：
INSERT INTO post_comment (post_id, review, id)
VALUES (1, 'Excellent book to understand Java Persistence', 1)
 
INSERT INTO post_comment (post_id, review, id)
VALUES (2, 'Must-read for Java developers', 2)
 
INSERT INTO post_comment (post_id, review, id)
VALUES (3, 'Five Stars', 3)
 
INSERT INTO post_comment (post_id, review, id)
VALUES (4, 'A great reference book', 4)

使用普通SQL的N+1查询问题
如果您选择post\u注释<
SELECT table1.*

SELECT table2.* WHERE SomeFkId = #

class House
{
    int Id { get; set; }
    string Address { get; set; }
    Person[] Inhabitants { get; set; }
}

class Person
{
    string Name { get; set; }
    int HouseId { get; set; }
}

Id Address      Name HouseId
1  22 Valley St Dave 1
1  22 Valley St John 1
1  22 Valley St Mike 1

Id Address
1  22 Valley St

SELECT * FROM Person WHERE HouseId = 1

Name    HouseId
Dave    1
John    1
Mike    1

***** Table: Supplier *****
+-----+-------------------+
| ID  |       NAME        |
+-----+-------------------+
|  1  |  Supplier Name 1  |
|  2  |  Supplier Name 2  |
|  3  |  Supplier Name 3  |
|  4  |  Supplier Name 4  |
+-----+-------------------+

***** Table: Product *****
+-----+-----------+--------------------+-------+------------+
| ID  |   NAME    |     DESCRIPTION    | PRICE | SUPPLIERID |
+-----+-----------+--------------------+-------+------------+
|1    | Product 1 | Name for Product 1 |  2.0  |     1      |
|2    | Product 2 | Name for Product 2 | 22.0  |     1      |
|3    | Product 3 | Name for Product 3 | 30.0  |     2      |
|4    | Product 4 | Name for Product 4 |  7.0  |     3      |
+-----+-----------+--------------------+-------+------------+

// It takes Select fetch mode as a default
Query query = session.createQuery( "from Product p");
List list = query.list();
// Supplier is being accessed
displayProductsListWithSupplierName(results);

select ... various field names ... from PRODUCT
select ... various field names ... from SUPPLIER where SUPPLIER.id=?
select ... various field names ... from SUPPLIER where SUPPLIER.id=?
select ... various field names ... from SUPPLIER where SUPPLIER.id=?

for p in person:
    print p.car.colour

  select * from people_car_colour; # this is a view or sql function

  p.id | p.name | p.telno | car.id | car.type | car.colour
  -----+--------+---------+--------+----------+-----------
  2    | jones  | 2145    | 77     | ford     | red
  2    | jones  | 2145    | 1012   | toyota   | blue
  16   | ashby  | 124     | 99     | bmw      | yellow

for p in people:
    print p.car.colour # no more car queries

INSERT INTO temp_ids 
    (product_id, batch_id)
    (SELECT p.product_id, ? 
    FROM product p ORDER BY p.product_id
    LIMIT ? OFFSET ?);

@Rule
public final QueryCounter queryCounter = new QueryCounter();

@Expectation(atMost = 3)
@Test
public void testInvokingDatabase() {
    // your JDBC or JPA code
}

INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 1', 1)
 
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 2', 2)
 
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 3', 3)
 
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 4', 4)

INSERT INTO post_comment (post_id, review, id)
VALUES (1, 'Excellent book to understand Java Persistence', 1)
 
INSERT INTO post_comment (post_id, review, id)
VALUES (2, 'Must-read for Java developers', 2)
 
INSERT INTO post_comment (post_id, review, id)
VALUES (3, 'Five Stars', 3)
 
INSERT INTO post_comment (post_id, review, id)
VALUES (4, 'A great reference book', 4)

List<Tuple> comments = entityManager.createNativeQuery("""
    SELECT
        pc.id AS id,
        pc.review AS review,
        pc.post_id AS postId
    FROM post_comment pc
    """, Tuple.class)
.getResultList();

for (Tuple comment : comments) {
    String review = (String) comment.get("review");
    Long postId = ((Number) comment.get("postId")).longValue();
 
    String postTitle = (String) entityManager.createNativeQuery("""
        SELECT
            p.title
        FROM post p
        WHERE p.id = :postId
        """)
    .setParameter("postId", postId)
    .getSingleResult();
 
    LOGGER.info(
        "The Post '{}' got this review '{}'",
        postTitle,
        review
    );
}

SELECT
    pc.id AS id,
    pc.review AS review,
    pc.post_id AS postId
FROM post_comment pc
 
SELECT p.title FROM post p WHERE p.id = 1
-- The Post 'High-Performance Java Persistence - Part 1' got this review
-- 'Excellent book to understand Java Persistence'
    
SELECT p.title FROM post p WHERE p.id = 2
-- The Post 'High-Performance Java Persistence - Part 2' got this review
-- 'Must-read for Java developers'
     
SELECT p.title FROM post p WHERE p.id = 3
-- The Post 'High-Performance Java Persistence - Part 3' got this review
-- 'Five Stars'
     
SELECT p.title FROM post p WHERE p.id = 4
-- The Post 'High-Performance Java Persistence - Part 4' got this review
-- 'A great reference book'

List<Tuple> comments = entityManager.createNativeQuery("""
    SELECT
        pc.id AS id,
        pc.review AS review,
        p.title AS postTitle
    FROM post_comment pc
    JOIN post p ON pc.post_id = p.id
    """, Tuple.class)
.getResultList();
 
for (Tuple comment : comments) {
    String review = (String) comment.get("review");
    String postTitle = (String) comment.get("postTitle");
 
    LOGGER.info(
        "The Post '{}' got this review '{}'",
        postTitle,
        review
    );
}

@Entity(name = "Post")
@Table(name = "post")
public class Post {
 
    @Id
    private Long id;
 
    private String title;
 
    //Getters and setters omitted for brevity
}
 
@Entity(name = "PostComment")
@Table(name = "post_comment")
public class PostComment {
 
    @Id
    private Long id;
 
    @ManyToOne
    private Post post;
 
    private String review;
 
    //Getters and setters omitted for brevity
}

@ManyToOne
private Post post;

List<PostComment> comments = entityManager
.createQuery("""
    select pc
    from PostComment pc
    """, PostComment.class)
.getResultList();

SELECT 
    pc.id AS id1_1_, 
    pc.post_id AS post_id3_1_, 
    pc.review AS review2_1_ 
FROM 
    post_comment pc

SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 1
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 2
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 3
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 4

List<PostComment> comments = entityManager.createQuery("""
    select pc
    from PostComment pc
    join fetch pc.post p
    """, PostComment.class)
.getResultList();

for(PostComment comment : comments) {
    LOGGER.info(
        "The Post '{}' got this review '{}'", 
        comment.getPost().getTitle(), 
        comment.getReview()
    );
}

SELECT 
    pc.id as id1_1_0_, 
    pc.post_id as post_id3_1_0_, 
    pc.review as review2_1_0_, 
    p.id as id1_0_1_, 
    p.title as title2_0_1_ 
FROM 
    post_comment pc 
INNER JOIN 
    post p ON pc.post_id = p.id
    
-- The Post 'High-Performance Java Persistence - Part 1' got this review 
-- 'Excellent book to understand Java Persistence'

-- The Post 'High-Performance Java Persistence - Part 2' got this review 
-- 'Must-read for Java developers'

-- The Post 'High-Performance Java Persistence - Part 3' got this review 
-- 'Five Stars'

-- The Post 'High-Performance Java Persistence - Part 4' got this review 
-- 'A great reference book'

@ManyToOne(fetch = FetchType.LAZY)
private Post post;

List<PostComment> comments = entityManager
.createQuery("""
    select pc
    from PostComment pc
    """, PostComment.class)
.getResultList();

SELECT 
    pc.id AS id1_1_, 
    pc.post_id AS post_id3_1_, 
    pc.review AS review2_1_ 
FROM 
    post_comment pc

for(PostComment comment : comments) {
    LOGGER.info(
        "The Post '{}' got this review '{}'", 
        comment.getPost().getTitle(), 
        comment.getReview()
    );
}

SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 1
-- The Post 'High-Performance Java Persistence - Part 1' got this review 
-- 'Excellent book to understand Java Persistence'

SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 2
-- The Post 'High-Performance Java Persistence - Part 2' got this review 
-- 'Must-read for Java developers'

SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 3
-- The Post 'High-Performance Java Persistence - Part 3' got this review 
-- 'Five Stars'

SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 4
-- The Post 'High-Performance Java Persistence - Part 4' got this review 
-- 'A great reference book'

List<PostComment> comments = entityManager.createQuery("""
    select pc
    from PostComment pc
    join fetch pc.post p
    """, PostComment.class)
.getResultList();

for(PostComment comment : comments) {
    LOGGER.info(
        "The Post '{}' got this review '{}'", 
        comment.getPost().getTitle(), 
        comment.getReview()
    );
}

<dependency>
    <groupId>com.vladmihalcea</groupId>
    <artifactId>db-util</artifactId>
    <version>${db-util.version}</version>
</dependency>

SQLStatementCountValidator.reset();

List<PostComment> comments = entityManager.createQuery("""
    select pc
    from PostComment pc
    """, PostComment.class)
.getResultList();

SQLStatementCountValidator.assertSelectCount(1);

SELECT 
    pc.id as id1_1_, 
    pc.post_id as post_id3_1_, 
    pc.review as review2_1_ 
FROM 
    post_comment pc

SELECT p.id as id1_0_0_, p.title as title2_0_0_ FROM post p WHERE p.id = 1

SELECT p.id as id1_0_0_, p.title as title2_0_0_ FROM post p WHERE p.id = 2


-- SQLStatementCountMismatchException: Expected 1 statement(s) but recorded 3 instead!

2020-10-22 18:41:43.236 DEBUG 14913 --- [           main] c.a.j.core.report.ReportGenerator        :
    ROOT
        com.adgadev.jplusone.test.domain.bookshop.BookshopControllerTest.shouldGetBookDetailsLazily(BookshopControllerTest.java:65)
        com.adgadev.jplusone.test.domain.bookshop.BookshopController.getSampleBookUsingLazyLoading(BookshopController.java:31)
        com.adgadev.jplusone.test.domain.bookshop.BookshopService.getSampleBookDetailsUsingLazyLoading [PROXY]
            SESSION BOUNDARY
                OPERATION [IMPLICIT]
                    com.adgadev.jplusone.test.domain.bookshop.BookshopService.getSampleBookDetailsUsingLazyLoading(BookshopService.java:35)
                    com.adgadev.jplusone.test.domain.bookshop.Author.getName [PROXY]
                    com.adgadev.jplusone.test.domain.bookshop.Author [FETCHING ENTITY]
                        STATEMENT [READ]
                            select [...] from
                                author author0_
                                left outer join genre genre1_ on author0_.genre_id=genre1_.id
                            where
                                author0_.id=1
                OPERATION [IMPLICIT]
                    com.adgadev.jplusone.test.domain.bookshop.BookshopService.getSampleBookDetailsUsingLazyLoading(BookshopService.java:36)
                    com.adgadev.jplusone.test.domain.bookshop.Author.countWrittenBooks(Author.java:53)
                    com.adgadev.jplusone.test.domain.bookshop.Author.books [FETCHING COLLECTION]
                        STATEMENT [READ]
                            select [...] from
                                book books0_
                            where
                                books0_.author_id=1