Sql 蜂巢连接选择非常慢

Sql 蜂巢连接选择非常慢,sql,hadoop,hive,hiveql,Sql,Hadoop,Hive,Hiveql,嗨,我有两个表:用户信息,ip地址,一个是50000,另一个是100000。 现在需要使用user表的ip来检查属性,将ip转换为int并将interval与ip_位置进行比较 我的配置单元版本是3.0.0,此版本没有索引 ip_位置: 在pg中,此操作非常快: set search_path=res; select * from( select ip, (split_part(ip,'.',1)::bigint*256*256*256 +split_part(ip,'.',2)::bigin

嗨,我有两个表:用户信息,ip地址,一个是50000,另一个是100000。 现在需要使用user表的ip来检查属性,将ip转换为int并将interval与ip_位置进行比较

我的配置单元版本是3.0.0,此版本没有索引

ip_位置:

在pg中,此操作非常快:

set search_path=res;
select * from(
select ip,
(split_part(ip,'.',1)::bigint*256*256*256
+split_part(ip,'.',2)::bigint*256*256
+split_part(ip,'.',3)::bigint*256
+split_part(ip,'.',4)::bigint)::int8 as ipvalue
from user_info) t1
left join ip_location t2 on 
ipv4_val_begin=(select max(ipv4_val_begin) from ip_location where ipv4_val_begin <= ipvalue);
设置搜索路径=res;
从中选择*(
选择ip,
(拆分部分(ip,'.',1)::bigint*256*256*256
+拆分部分(ip,'.',2)::bigint*256*256
+拆分部分(ip,'.',3)::bigint*256
+拆分部分(ip,'.',4)::bigint::int8作为ipvalue
来自用户信息)t1
左连接ip_位置t2打开

ipv4\u val\u begin=(选择最大值(ipv4\u val\u begin)从ipv4的ip地址开始我尝试了查看和行组索引,但速度没有提高多少。我想问一下如何使用hive来加速ip地址范围,spark上的hive速度也很慢。

hive版本是否支持
with
子句?
with
子句不能提高速度d、 这仍然需要一天的时间,博士后只需要20秒
select ip,
t2.location_country
cast(split(ip,"\\.")[0] as bigint)*256*256*256
+cast(split(ip,"\\.")[0] as bigint)*256*256
+cast(split(ip,"\\.")[0] as bigint)*256
+cast(split(ip,"\\.")[0] as bigint) as ipvalue
from source.v_dm_vip_user t1
left join res.ip_location t2 on 
ipv4_val_begin=(select max(ipv4_val_begin) from res.ip_location where ipv4_val_begin <= ipvalue);
select ip,
t2.location_country
cast(split(ip,"\\.")[0] as bigint)*256*256*256
+cast(split(ip,"\\.")[0] as bigint)*256*256
+cast(split(ip,"\\.")[0] as bigint)*256
+cast(split(ip,"\\.")[0] as bigint) as ipvalue
from source.v_dm_vip_user t1
left join res.ip_location t2 on 
cast(split(ip,"\\.")[0] as bigint)*256*256*256
+cast(split(ip,"\\.")[0] as bigint)*256*256
+cast(split(ip,"\\.")[0] as bigint)*256
+cast(split(ip,"\\.")[0] as bigint) > ipv4_val_begin
and 
cast(split(ip,"\\.")[0] as bigint)*256*256*256
+cast(split(ip,"\\.")[0] as bigint)*256*256
+cast(split(ip,"\\.")[0] as bigint)*256
+cast(split(ip,"\\.")[0] as bigint) < ipv4_val_end;