Arrays 如何在配置单元中将多个列值转换为行?
代码: 错误: 失败:语义异常[错误10011]:无效的函数数组\u索引Arrays 如何在配置单元中将多个列值转换为行?,arrays,hadoop,hive,explode,Arrays,Hadoop,Hive,Explode,代码: 错误: 失败:语义异常[错误10011]:无效的函数数组\u索引 在这里,我在单个列中有多个值,我需要将其转换为前面提到的行输出。Explode是hive中提供的UDTF,您可以使用它将数据从列拆分为行 select ID, array_index( COLUMN1_arr, n ) as COLUMN1, array_index( COLUMN2_arr, n ) as COLUMN2 from sample lateral view numeric_range(size(COLUMN
在这里,我在单个列中有多个值,我需要将其转换为前面提到的行输出。Explode是hive中提供的UDTF,您可以使用它将数据从列拆分为行
select ID,
array_index( COLUMN1_arr, n ) as COLUMN1,
array_index( COLUMN2_arr, n ) as COLUMN2
from sample
lateral view numeric_range(size(COLUMN1_arr)) n1 as n;
Explode是配置单元中提供的UDTF,您可以使用它将数据从列拆分为行
select ID,
array_index( COLUMN1_arr, n ) as COLUMN1,
array_index( COLUMN2_arr, n ) as COLUMN2
from sample
lateral view numeric_range(size(COLUMN1_arr)) n1 as n;
纯香草蜂箱溶液,不含砖砌UDF 演示: 结果:
with
input as ( ---------------Input dataset
select stack(2,
1, array('M','S','E','T'), array(1,2,3,4), array(5,6,7),
2, array('A','B','C'), array(6,5,8,7,9,1), array( 2,4,3,0,1)
) as (ID,COLUMN1,COLUMN2,COLUMN3)
),
--explode each array and FULL join them
c1 as (
select i.id, v.column1, p
from input i
lateral view posexplode(i.COLUMN1) v as p,column1
),
c2 as (
select i.id, v.column2, p
from input i
lateral view posexplode(i.COLUMN2) v as p,column2
),
c3 as (
select i.id, v.column3, p
from input i
lateral view posexplode(i.COLUMN3) v as p,column3
)
--FULL JOIN
select coalesce(c1.id,c2.id,c3.id) id, c1.column1, c2.column2, c3.column3
from c1
full join c2 on c1.id=c2.id and c1.p=c2.p
full join c3 on nvl(c1.id,c2.id)=c3.id and nvl(c1.p,c2.p)=c3.p --note NVL usage
;
纯香草蜂箱溶液,不含砖砌UDF 演示: 结果:
with
input as ( ---------------Input dataset
select stack(2,
1, array('M','S','E','T'), array(1,2,3,4), array(5,6,7),
2, array('A','B','C'), array(6,5,8,7,9,1), array( 2,4,3,0,1)
) as (ID,COLUMN1,COLUMN2,COLUMN3)
),
--explode each array and FULL join them
c1 as (
select i.id, v.column1, p
from input i
lateral view posexplode(i.COLUMN1) v as p,column1
),
c2 as (
select i.id, v.column2, p
from input i
lateral view posexplode(i.COLUMN2) v as p,column2
),
c3 as (
select i.id, v.column3, p
from input i
lateral view posexplode(i.COLUMN3) v as p,column3
)
--FULL JOIN
select coalesce(c1.id,c2.id,c3.id) id, c1.column1, c2.column2, c3.column3
from c1
full join c2 on c1.id=c2.id and c1.p=c2.p
full join c3 on nvl(c1.id,c2.id)=c3.id and nvl(c1.p,c2.p)=c3.p --note NVL usage
;
你曾经测试过这样的查询吗?在这种情况下,多个侧面视图将生成笛卡尔产品。您是否测试过这种查询?在这种情况下,多个侧视图将产生笛卡尔积
OK
id column1 column2 column3
1 M 1 5
1 S 2 6
1 E 3 7
1 T 4 NULL
2 A 6 2
2 B 5 4
2 C 8 3
2 NULL 7 0
2 NULL 9 1
2 NULL 1 NULL