Hive 配置单元解释计划未显示分区
我有一个表,其中包含2.51亿条记录,大小为2.5gb。 我在两列上创建了一个分区,我正在谓词中执行该分区。 但是解释计划没有显示它正在读取分区,即使我已经分区了。通过从分区列中选择,我将插入到另一个表中 我在谓词中提到条件是否有特定的顺序? 我应该如何提高绩效Hive 配置单元解释计划未显示分区,hive,hadoop-partitioning,Hive,Hadoop Partitioning,我有一个表,其中包含2.51亿条记录,大小为2.5gb。 我在两列上创建了一个分区,我正在谓词中执行该分区。 但是解释计划没有显示它正在读取分区,即使我已经分区了。通过从分区列中选择,我将插入到另一个表中 我在谓词中提到条件是否有特定的顺序? 我应该如何提高绩效 explain SELECT '123' AS run_se
explain
SELECT
'123' AS run_session_id
, tbl1.transaction_id
, tbl1.src_transaction_id
, tbl1.transaction_created_epoch_time
, tbl1.currency
, tbl1.event_type
, tbl1.event_sub_type
, tbl1.estimated_total_cost
, tbl1.actual_total_cost
, tbl1.tfc_export_created_epoch_time
, tbl1.authorizer
, tbl1.acquirer
, tbl1.processor
, tbl1.company_code
, tbl1.country_of_account
, tbl1.merchant_id
, tbl1.client_id
, tbl1.ft_id
, tbl1.transaction_created_date
, tbl1.event_pst_time
, tbl1.extract_id_seq
, tbl1.src_type
, ROW_NUMBER() OVER(PARTITION by tbl1.transaction_id ORDER BY tbl1.event_pst_time DESC) AS seq_num -- while writing back to the pfit events table, write each event so that event_pst_time populates in right way
FROM db.xx_events tbl1 --<hiveFinalDB>-- -- DB variables wont work, so need to change the DB accrodingly for testing and PROD deployment
WHERE id_seq >= 215
AND id_seq <= 275
AND vent in('SPT','PNR','PNE','PNER','ACT','NTE');
现在我如何提高性能。分区列是id_seq,vent解释依赖项选择
演示
输入分区:[
{分区名称:本地_db@mytable@dt=2017-06-20},{分区名称:本地_db@mytable@dt=2017-06-21},{分区名称:本地_db@mytable@dt=2017-06-22}]}
create table mytable (i int)
partitioned by (dt date)
;
alter table mytable add
partition (dt=date '2017-06-18')
partition (dt=date '2017-06-19')
partition (dt=date '2017-06-20')
partition (dt=date '2017-06-21')
partition (dt=date '2017-06-22')
;
explain dependency
select *
from mytable
where dt >= date '2017-06-20'
;
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Explain |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| {"input_tables":[{"tablename":"local_db@mytable","tabletype":"MANAGED_TABLE"}],"input_partitions":[{"partitionName":"local_db@mytable@dt=2017-06-20"},{"partitionName":"local_db@mytable@dt=2017-06-21"},{"partitionName":"local_db@mytable@dt=2017-06-22"}]} |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+