Fatal编程技术网
  • memory/
  • Memory 在RDS服务器上创建临时表需要很长时间

Memory 在RDS服务器上创建临时表需要很长时间

memory

Memory 在RDS服务器上创建临时表需要很长时间,memory,innodb,performance-testing,mysql-5.7,sql-tuning,Memory,Innodb,Performance Testing,Mysql 5.7,Sql Tuning,下面的查询创建临时表需要很长时间，它只有“228000”条不同的记录 DECLARE todate,fromdate DATETIME; SET fromdate=DATE_SUB(UTC_TIMESTAMP(),INTERVAL 2 DAY); SET todate=DATE_ADD(UTC_TIMESTAMP(),INTERVAL 14 DAY); SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED; DROP TEMPOR

下面的查询创建临时表需要很长时间，它只有“228000”条不同的记录

DECLARE todate,fromdate DATETIME;

SET fromdate=DATE_SUB(UTC_TIMESTAMP(),INTERVAL 2 DAY);
SET todate=DATE_ADD(UTC_TIMESTAMP(),INTERVAL 14 DAY);


SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;

 DROP TEMPORARY TABLE IF EXISTS tempabc;


SET max_heap_table_size = 1024*1024*1024;

CREATE TEMPORARY TABLE IF NOT EXISTS tempabc
-- (index using BTREE(id))
ENGINE=MEMORY

AS
( 
    SELECT SQL_NO_CACHE DISTINCT id
    FROM abc 
    WHERE StartTime BETWEEN fromdate AND todate
);
我已经在“startTime”coulmn上创建了索引，但创建表需要20秒。请帮助我减少创建时间

更多信息：-

我之前更改了查询，我使用“tempabc”临时表来获取输出，现在我使用IN子句而不是临时表，现在执行需要12秒，但仍然比预期的时间长

更早的时间（需要20-30秒）

现在（持续12-14秒）

但我们需要达到3-5秒的执行时间

这是我的解释输出

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: abc
   partitions: NULL
         type: index
possible_keys: ix_starttime_id,IDX_Start_time,IX_id_starttime,IX_id_starttime_prgsvcid
          key: IX_id_starttime
      key_len: 163
          ref: NULL
         rows: 18779876
     filtered: 1.27
        Extra: Using where; Using index; Using temporary; Using filesort; LooseScan
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: p
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY,IX_seriesid
          key: PRIMARY
      key_len: 152
          ref: onconnectdb.abc.ID
         rows: 1
     filtered: 100.00
        Extra: Using where
以JSON格式解释

EXPLAIN: {
  "query_block": {
    "select_id": 1,
    "cost_info": {
      "query_cost": "10139148.44"
    },
    "grouping_operation": {
      "using_temporary_table": true,
      "using_filesort": true,
      "cost_info": {
        "sort_cost": "1.00"
      },
      "nested_loop": [
        {
          "table": {
            "table_name": "abc",
            "access_type": "index",
            "possible_keys": [
              "ix_starttime_tmsid",
              "IDX_Start_time",
              "IX_id_starttime",
              "IX_id_starttime_prgsvcid"
            ],
            "key": "IX_id_starttime",
            "used_key_parts": [
              "ID",
              "StartTime",
              "EndTime"
            ],
            "key_length": "163",
            "rows_examined_per_scan": 19280092,
            "rows_produced_per_join": 264059,
            "filtered": "1.37",
            "using_index": true,
            "loosescan": true,
            "cost_info": {
              "read_cost": "393472.45",
              "eval_cost": "52812.00",
              "prefix_cost": "446284.45",
              "data_read_per_join": "2G"
            },
            "used_columns": [
              "ID",
              "StartTime"
            ],
            "attached_condition": "(`onconnectdb`.`abc`.`StartTime` between <cache>(fromdate@1) and <cache>(todate@0))"
          }
        },
        {
          "table": {
            "table_name": "p",
            "access_type": "eq_ref",
            "possible_keys": [
              "PRIMARY",
              "IX_seriesid"
            ],
            "key": "PRIMARY",
            "used_key_parts": [
              "ID"
            ],
            "key_length": "152",
            "ref": [
              "onconnectdb.abc.ID"
            ],
            "rows_examined_per_scan": 1,
            "rows_produced_per_join": 1,
            "filtered": "100.00",
            "cost_info": {
              "read_cost": "9640051.00",
              "eval_cost": "0.20",
              "prefix_cost": "10139147.44",
              "data_read_per_join": "2K"
            },
            "used_columns": [
              "ID",
              "xyzID",
              "IsGeneric"
            ],
            "attached_condition": "(ifnull(`onconnectdb`.`p`.`IsGeneric`,0) = 0)"
          }
        }
      ]
    }
  }
}

解释：{
“查询块”：{
“选择id”：1，
“成本信息”：{
“查询成本”：“10139148.44”
},
“分组操作”：{
“使用临时表格”：真，
“使用_filesort”：true，
“成本信息”：{
“分拣成本”：“1.00”
},
“嵌套的_循环”：[
{
“表格”：{
“表格名称”：“abc”，
“访问类型”：“索引”，
“可能的_键”：[
“ix_starttime_tmsid”，
“IDX\u开始时间”，
“IX_id_starttime”，
“IX_id_starttime_prgsvcid”
],
“密钥”：“IX_id_starttime”，
“使用过的关键部件”：[
“身份证”，
“开始时间”，
“结束时间”
],
“密钥长度”：“163”，
“每次扫描检查的行数”：19280092，
“每个联接产生的行数”：264059，
“过滤”：“1.37”，
“使用指数”：正确，
“loosescan”：没错，
“成本信息”：{
“阅读成本”：“393472.45”，
“评估成本”：“52812.00”，
“前缀成本”：“446284.45”，
“每个联接的数据读取”：“2G”
},
“已用列”：[
“身份证”，
“开始时间”
],
“附加条件”：“（`onconnectdb`.`abc`.`StartTime`(fromdate@1)及(todate@0))"
}
},
{
“表格”：{
“表格名称”：“p”，
“访问类型”：“eq\u ref”，
“可能的_键”：[
“主要”，
“IX_系列ID”
],
“密钥”：“主要”，
“使用过的关键部件”：[
“ID”
],
“密钥长度”：“152”，
“参考”：[
“onconnectdb.abc.ID”
],
“每次扫描检查的行数”：1，
“每个联接产生的行数”：1，
“过滤”：“100.00”，
“成本信息”：{
“读取成本”：“9640051.00”，
“评估成本”：“0.20”，
“前缀成本”：“10139147.44”，
“每个联接的数据读取”：“2K”
},
“已用列”：[
“身份证”，
“xyzID”，
“是通用的”
],
“附加的条件”：“（ifnull（`onconnectdb`.`p`.`IsGeneric`，0）=0）
}
}
]
}
}
}
请建议。

abc中有多少行？
abc
是否有
索引（StartTime，id）
？@RickJames，abc表共有200万条记录，其在（StartTime，id）列上的索引是
id
的
主键
？如果是这样，则不需要使用DISTINCT。请提供
解释选择。。。todate
@RickJames，谢谢，我刚刚添加了有关我的查询的更多信息。请提供
EXPLAIN FORMAT=JSON SELECT…
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: abc
   partitions: NULL
         type: index
possible_keys: ix_starttime_id,IDX_Start_time,IX_id_starttime,IX_id_starttime_prgsvcid
          key: IX_id_starttime
      key_len: 163
          ref: NULL
         rows: 18779876
     filtered: 1.27
        Extra: Using where; Using index; Using temporary; Using filesort; LooseScan
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: p
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY,IX_seriesid
          key: PRIMARY
      key_len: 152
          ref: onconnectdb.abc.ID
         rows: 1
     filtered: 100.00
        Extra: Using where

EXPLAIN: {
  "query_block": {
    "select_id": 1,
    "cost_info": {
      "query_cost": "10139148.44"
    },
    "grouping_operation": {
      "using_temporary_table": true,
      "using_filesort": true,
      "cost_info": {
        "sort_cost": "1.00"
      },
      "nested_loop": [
        {
          "table": {
            "table_name": "abc",
            "access_type": "index",
            "possible_keys": [
              "ix_starttime_tmsid",
              "IDX_Start_time",
              "IX_id_starttime",
              "IX_id_starttime_prgsvcid"
            ],
            "key": "IX_id_starttime",
            "used_key_parts": [
              "ID",
              "StartTime",
              "EndTime"
            ],
            "key_length": "163",
            "rows_examined_per_scan": 19280092,
            "rows_produced_per_join": 264059,
            "filtered": "1.37",
            "using_index": true,
            "loosescan": true,
            "cost_info": {
              "read_cost": "393472.45",
              "eval_cost": "52812.00",
              "prefix_cost": "446284.45",
              "data_read_per_join": "2G"
            },
            "used_columns": [
              "ID",
              "StartTime"
            ],
            "attached_condition": "(`onconnectdb`.`abc`.`StartTime` between <cache>(fromdate@1) and <cache>(todate@0))"
          }
        },
        {
          "table": {
            "table_name": "p",
            "access_type": "eq_ref",
            "possible_keys": [
              "PRIMARY",
              "IX_seriesid"
            ],
            "key": "PRIMARY",
            "used_key_parts": [
              "ID"
            ],
            "key_length": "152",
            "ref": [
              "onconnectdb.abc.ID"
            ],
            "rows_examined_per_scan": 1,
            "rows_produced_per_join": 1,
            "filtered": "100.00",
            "cost_info": {
              "read_cost": "9640051.00",
              "eval_cost": "0.20",
              "prefix_cost": "10139147.44",
              "data_read_per_join": "2K"
            },
            "used_columns": [
              "ID",
              "xyzID",
              "IsGeneric"
            ],
            "attached_condition": "(ifnull(`onconnectdb`.`p`.`IsGeneric`,0) = 0)"
          }
        }
      ]
    }
  }
}