Fatal编程技术网
  • mysql/
  • Mysql 我的SQL非常慢，而且我不'；我不知道该怎么办

Mysql 我的SQL非常慢，而且我不'；我不知道该怎么办

mysql sql

Mysql 我的SQL非常慢，而且我不'；我不知道该怎么办,mysql,sql,Mysql,Sql,我是一个SQL新手，但我同意在SQL中进行Faceted搜索，因为我已经使用其他数据库进行了多次搜索 我在几天内就完成了，但当我尝试使用大数据集时，这需要花费很长时间，我已经尝试优化它一周了，但仍然非常慢 以下是两个表格： record_facet（带有facet的表，例如{id:1，facet_name:color，value:blue}） 这张桌子有2米多行 +------------+--------------+------+-----+---------+-------+ | Fiel

我是一个SQL新手，但我同意在SQL中进行Faceted搜索，因为我已经使用其他数据库进行了多次搜索

我在几天内就完成了，但当我尝试使用大数据集时，这需要花费很长时间，我已经尝试优化它一周了，但仍然非常慢

以下是两个表格：

record_facet（带有facet的表，例如{id:1，facet_name:color，value:blue}） 这张桌子有2米多行

+------------+--------------+------+-----+---------+-------+
| Field      | Type         | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+-------+
| rec_id     | int(11)      | NO   | MUL | NULL    |       |
| facet_name | varchar(50)  | NO   | MUL | NULL    |       |
| value      | varchar(255) | NO   |     | NULL    |       |
+------------+--------------+------+-----+---------+-------+
表“记录”是您通过单击或松开刻面找到的实际记录， 它有很多列（正文、图片、url），但我只使用id，所以我只在这里列出id：

记录

+--------------------------------+------------+------+-----+---------+-------+
| id                             | bigint(20) | YES  | MUL | NULL    |       |
+--------------------------------+------------+------+-----+---------+-------+
以下是我的索引：

SHOW INDEX FROM record;
+--------+------------+--------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table  | Non_unique | Key_name     | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------+------------+--------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| record |          1 | ix_record_id |            1 | id          | A         |      177054 |     NULL | NULL   | YES  | BTREE      |         |               |
+--------+------------+--------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

SHOW INDEX FROM record_facet
+--------------+------------+-------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table        | Non_unique | Key_name                | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------------+------------+-------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| record_facet |          1 | ix_facet_name_and_value |            1 | facet_name  | A         |          10 |     NULL | NULL   |      | BTREE      |         |               |
| record_facet |          1 | ix_facet_name_and_value |            2 | value       | A         |        3527 |     NULL | NULL   |      | BTREE      |         |               |
| record_facet |          1 | ix_rec_id               |            1 | rec_id      | A         |      423773 |     NULL | NULL   |      | BTREE      |         |               |
+--------------+------------+-------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
我有两个疑问：

查询以获取面： （这是慢的一个）

这是获取结果的查询： （不那么慢）

以下是解释：

SLOW QUERY:
+------+-------------+------------+-------+-----------------------------------+-------------------------+---------+----------------------------------------+--------+---------------------------------------------------------------------+
| id   | select_type | table      | type  | possible_keys                     | key                     | key_len | ref                                    | rows   | Extra                                                               |
+------+-------------+------------+-------+-----------------------------------+-------------------------+---------+----------------------------------------+--------+---------------------------------------------------------------------+
|    1 | PRIMARY     | rf         | index | ix_rec_id                         | ix_facet_name_and_value | 309     | NULL                                   | 296638 |                                                                     |
|    1 | PRIMARY     | <derived2> | ref   | key0                              | key0                    | 9       | sqlse_test_crescentbconflate.rf.rec_id |     10 | Using where                                                         |
|    2 | DERIVED     | f          | ref   | ix_facet_name_and_value,ix_rec_id | ix_facet_name_and_value | 52      | const                                  | 833738 | Using index condition; Using where; Using temporary; Using filesort |
|    2 | DERIVED     | r          | ref   | ix_record_id                      | ix_record_id            | 9       | sqlse_test_crescentbconflate.f.rec_id  |      1 | Using where; Using index                                            |
+------+-------------+------------+-------+-----------------------------------+-------------------------+---------+----------------------------------------+--------+---------------------------------------------------------------------+


THE OTHER QUERY: 

+------+-------------+-------+-------+-----------------------------------+--------------+---------+-----------------------------------+------+------------------------------------+
| id   | select_type | table | type  | possible_keys                     | key          | key_len | ref                               | rows | Extra                              |
+------+-------------+-------+-------+-----------------------------------+--------------+---------+-----------------------------------+------+------------------------------------+
|    1 | SIMPLE      | r     | index | ix_record_id                      | ix_record_id | 9       | NULL                              |    5 | Using where                        |
|    1 | SIMPLE      | f     | ref   | ix_facet_name_and_value,ix_rec_id | ix_rec_id    | 4       | sqlse_test_crescentbconflate.r.id |    7 | Using index condition; Using where |
+------+-------------+-------+-------+-----------------------------------+--------------+---------+-----------------------------------+------+------------------------------------+

慢速查询：
+------+-------------+------------+-------+-----------------------------------+-------------------------+---------+----------------------------------------+--------+---------------------------------------------------------------------+
|id |选择|类型|类型|可能的|键|键|列|参考|行|额外|
+------+-------------+------------+-------+-----------------------------------+-------------------------+---------+----------------------------------------+--------+---------------------------------------------------------------------+
|1 |主| rf |索引| ix |记录| id | ix |刻面|名称|和|值| 309 |空| 296638 ||
|1 |主| |参考|键0 |键0 | 9 | sqlse | U测试| U crescentbconflate.rf.rec | U id | 10 |使用where|
|2 |导出| f | ref | ix | facet | name |和|值，ix | rec | id | ix | facet | name |和| u值| 52 | const | 833738 |；在何处使用；使用临时设备；使用文件排序|
|2 |派生| r |参考| ix |记录| id | ix |记录| id | 9 | sqlse |测试| crescentbconflate.f.rec | id | 1 |使用where；使用索引|
+------+-------------+------------+-------+-----------------------------------+-------------------------+---------+----------------------------------------+--------+---------------------------------------------------------------------+
另一个问题：
+------+-------------+-------+-------+-----------------------------------+--------------+---------+-----------------------------------+------+------------------------------------+
|id |选择|类型|类型|可能的|键|键|列|参考|行|额外|
+------+-------------+-------+-------+-----------------------------------+--------------+---------+-----------------------------------+------+------------------------------------+
|1 |简单| r |索引| ix|U记录| id | ix|U记录| id | 9 |空| 5 |使用where|
|1 |简单| f |参考| ix |刻面|名称|和|值，ix | rec | id | ix | rec | id | 4 | sqlse |测试| crescentbconflate.r.id | 7 |使用索引条件；在哪里使用|
+------+-------------+-------+-------+-----------------------------------+--------------+---------+-----------------------------------+------+------------------------------------+
我只关心缓慢的查询。
非常感谢您的帮助，非常感谢

这是您的查询，从我的角度来看更具可读性：

SELECT rf.facet_name, rf.value, count(*) AS c
FROM record_facet rf INNER JOIN
     (SELECT r.id 
      FROM record r LEFT JOIN
           record_facet f
           ON rf.id = f.rec_id 
      WHERE ( f.facet_name = 'HasProp' AND f.value = 'PartNum' )  OR 
            ( f.facet_name = 'HasProp' AND f.value = 'Price' ) 
      GROUP BY r.id
      HAVING COUNT(r.id) = 2
     ) rs 
     ON rs.id = rf.rec_id
GROUP BY rf.facet_name, rf.value
LIMIT 10000;
这可以通过删除子查询中的
join
来简化。似乎没有必要：

SELECT rf.facet_name, rf.value, count(*) AS c
FROM record_facet rf INNER JOIN
     (SELECT f.rec_id
      FROM record_facet f
      WHERE f.facet_name = 'HasProp' AND f.value  IN ('PartNum', 'Price' ) 
      GROUP BY f.rec_id
      HAVING COUNT(*) = 2
     ) rs 
     ON rs.rec_id = rf.rec_id
GROUP BY rf.facet_name, rf.value
LIMIT 10000;
这个版本可以从
record\u facet（facet\u name，value）
以及
record\u facet（rec\u id，facet\u name，value）
的索引中获益

其次，这反过来似乎要求所有具有这两种价值的“方面”。所以，我倾向于这样写：

SELECT rf.facet_name, rf.value, count(*) AS c
FROM record_facet rf 
WHERE EXISTS (SELECT 1 
              FROM record_facet f2
              WHERE f2.rec_id = rf.rec_id AND
                    f2.facet_name = 'HasProp' AND f2.value = 'PartNum'
             ) AND
      EXISTS (SELECT 1 
              FROM record_facet f2
              WHERE f2.rec_id = rf.rec_id AND
                    f2.facet_name = 'HasProp' AND f2.value = 'Price'
             )
GROUP BY rf.facet_name, rf.value
LIMIT 10000;
为此，如果我们保证
（rec\u id，facet\u name，value）
（rec\u id，facet\u name，value）
元组在
记录facet
表中是唯一的，那么您需要在
记录facet（rec\u id，facet\u name，value）上建立一个复合索引，然后，查询
JOIN
操作将返回与来自Gordon答案的
EXISTS（相关子查询）
示例查询相同的结果


  SELECT rf.facet_name   AS `facet_name`
       , rf.value        AS `value`
       , COUNT(*)        AS `c`
    FROM record_facet rf
    JOIN record_facet pr
      ON pr.rec_id = rf.rec_id
     AND pr.facet_name = 'HasProp'
     AND pr.value      = 'Price'
    JOIN record_facet pn
      ON pn.rec_id = rf.rec_id
     AND pn.facet_name = 'HasProp'
     AND pn.value      = 'PartNum'
   GROUP
      BY rf.facet_name
       , rf.value
   LIMIT 10000



  SELECT rf.facet_name   AS `facet_name`
       , rf.value        AS `value`
       , COUNT(*)        AS `c`
    FROM record_facet rf
    JOIN record_facet pr
      ON pr.rec_id = rf.rec_id
     AND pr.facet_name = 'HasProp'
     AND pr.value      = 'Price'
    JOIN record_facet pn
      ON pn.rec_id = rf.rec_id
     AND pn.facet_name = 'HasProp'
     AND pn.value      = 'PartNum'
   GROUP
      BY rf.facet_name
       , rf.value
   LIMIT 10000


需要适当的覆盖指数来实现最佳性能


... UNIQUE INDEX ... ON facet_record (rec_id, value, facet_name)
... UNIQUE INDEX ... ON facet_record (facet_name, value, rec_id)




tl；dr

看起来我们假设您正在使用InnoDB，并且InnoDB内存结构（如缓冲池）的大小适当，MySQL没有陷入对磁盘的破坏。）

呼应戈登的评论。。。接下来呢

查询的期望返回值似乎是所有
facet\u name
和
value
的集合，与每个
record
相关联的
record\u facet
行
HasProp
PartNum
和
Price

原始查询包含一个条件
COUNT（r.id）=2
，这使得我们看起来似乎只想返回与
记录相关联的行，这些记录既有
PartNum
又有
Price

查询中的条件似乎取决于
（rec_id，'HasProp'，'PartNum'）
的唯一性。也就是说，如果我们有两（或更多）行
PartNum


... UNIQUE INDEX ... ON facet_record (rec_id, value, facet_name)
... UNIQUE INDEX ... ON facet_record (facet_name, value, rec_id)



  SELECT rf.facet_name   AS `facet_name`
       , rf.value        AS `value`
       , COUNT(*)        AS `c`
    FROM record_facet rf
    JOIN record_facet pr
      ON pr.rec_id = rf.rec_id
     AND pr.facet_name = 'HasProp'
     AND pr.value      = 'Price'
    JOIN record_facet pn
      ON pn.rec_id = rf.rec_id
     AND pn.facet_name = 'HasProp'
     AND pn.value      = 'PartNum'
   GROUP
      BY rf.facet_name
       , rf.value
   LIMIT 10000



 ... UNIQUE INDEX ... ON facet_record (rec_id, value, facet_name)
 ... UNIQUE INDEX ... ON facet_record (facet_name, value, rec_id)



  ... UNIQUE INDEX ... ON facet_record (value, facet_name, rec_id)