Hive 他使用Glue crawler为uf列创建了相同的模式,并且能够unest。我的文件中的一行看起来像{“prop1”:1,“uf”:[{“type”:“1browserId”,“name”:“value”:“1unknown”},{“type”:“2bro

Hive 他使用Glue crawler为uf列创建了相同的模式,并且能够unest。我的文件中的一行看起来像{“prop1”:1,“uf”:[{“type”:“1browserId”,“name”:“value”:“1unknown”},{“type”:“2bro,hive,aws-glue,presto,amazon-athena,aws-glue-data-catalog,Hive,Aws Glue,Presto,Amazon Athena,Aws Glue Data Catalog,他使用Glue crawler为uf列创建了相同的模式,并且能够unest。我的文件中的一行看起来像{“prop1”:1,“uf”:[{“type”:“1browserId”,“name”:“value”:“1unknown”},{“type”:“2browserId”,“name”:“value”:“2unknown”}为什么不通过spark展平这个数组,然后在athena中使用它呢?@Ilykisil我添加了示例CSVline@syadav我还不知道Spark,因此,尝试使用我所知道的东西来


他使用Glue crawler为
uf
列创建了相同的模式,并且能够
unest
。我的文件中的一行看起来像
{“prop1”:1,“uf”:[{“type”:“1browserId”,“name”:“value”:“1unknown”},{“type”:“2browserId”,“name”:“value”:“2unknown”}
为什么不通过spark展平这个数组,然后在athena中使用它呢?@Ilykisil我添加了示例CSVline@syadav我还不知道Spark,因此,尝试使用我所知道的东西来实现这一点,您能否提供完整的表DDL定义,即
SHOW CREATE table
,以及整个JSON文件的示例,而不仅仅是列
uf
?当我试图复制您的JSON文件时,我使用Glue crawler为
uf
列获得了相同的模式,并且能够
unest
。我的文件中的一行看起来像
{“prop1”:1,“uf”:[{“type”:“1browserId”,“name”:“value”:“1unknown”},{“type”:“2browserId”,“name”:“value”:“2unknown”}
为什么不通过spark展平这个数组,然后在athena中使用它呢?@Ilykisil我添加了示例CSVline@syadav我还不知道Spark,因此,出于某种原因,我尝试使用我所知道的东西来执行转换,转换为json在select中可以工作,但如果是视图的一部分,则无法工作。例如:
selectcast(5asjson)“test”
起作用,而| create view bla as selectcast(5asjson)“test”由于某种原因不起作用,cast to json在select中起作用,但如果是视图的一部分则不起作用。例如:
selectcast(5作为json)“test”可以工作,而| createview bla作为selectcast(5作为json)“test”不能工作
[
    {"type": "browserId", "name": "", "value": "unknown"},
    {"type": "campaign", "name": "", "value": "om_227dec0082a5"},
    {"type": "custom", "name": "2351350529", "value": "10148"},
    {"type": "custom", "name": "9501713387", "value": "true"},
    {"type": "custom", "name": "9517735577", "value": "true"},
    {"type": "custom", "name": "9507402548", "value": "true"},
    {"type": "custom", "name": "9733902068", "value": "true"}
]

[{type=[{"type": "browserId", "name": "", "value": "ff"}, name=null, value=null}]
{
    "Name": "uf",
    "Type": "array<struct<type:string,name:string,value:string>>"
}
timestamp   project_id  campaign_id experiment_id   variation_id    layer_holdback  audience_names  end_user_id uuid    session_id  snippet_revision    user_ip user_agent  user_engine user_engine_version referer global_holdback event_type  event_name  uf  active_views    event_features  event_metrics   event_uuid
1570326511  74971132    11089500404 11097730080 11078120202 false   []  oeu1535997971348r0.4399811351004357     AUTO    6540    5.91.170.0  Mozilla/5.0 (Linux; Android 7.0; SAMSUNG SM-G925F Build/NRD90M) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/9.2 Chrome/67.0.3396.87 Mobile Safari/537.36  js  0.128.0 https://www.zavamed.com/uk/account/ false   view_activated  10832783364 [{"type": "browserId", "name": "", "value": "unknown"}, {"type": "device", "name": "", "value": "mobile"}, {"type": "device_type", "name": "", "value": "phone"}, {"type": "referrer", "name": "", "value": "https:\/\/www.google.co.uk\/"}, {"type": "source_type", "name": "", "value": "campaign"}, {"type": "currentTimestamp", "name": "", "value": "-1631518596"}, {"type": "offset", "name": "", "value": "-60"}]    []  []  []  4926a5f1-bbb5-4553-9d0b-b26f773fa0f4

    timestamp   _unnested_column
1   1570326511  {"name":"","type":"browserId","value":"unknown"}
2   1570326511  {"name":"","type":"device","value":"mobile"}
3   1570326511  {"name":"","type":"device_type","value":"phone"}
4   1570326511  {"name":"","type":"referrer","value":"https://www.google.co.uk/"}
5   1570326511  {"name":"","type":"source_type","value":"campaign"}
6   1570326511  {"name":"","type":"currentTimestamp","value":"-1631518596"}
7   1570326511  {"name":"","type":"offset","value":"-60"}