编写SQL脚本，获取BigQuery中所有列的列表并将它们注入到查询中

我有一张这样的桌子：

[ID]  [Country_Region]  [Date1]  [Date2]  [Date3] ... [Date150]

[ID] [Country_Region] [Date.ColumnName] [Date.Value] 
-----------------------------------------------------
 1    China           _1_22_20           12
 2    China           _1_23_20           34
 [3   China           _1_24_20           54] <----- I want this row and all the next ones to appear as well

我在BigQuery中编写了一个SQL查询，返回如下数据集：

[ID]  [Country_Region]  [Date1]  [Date2]  [Date3] ... [Date150]

[ID] [Country_Region] [Date.ColumnName] [Date.Value] 
-----------------------------------------------------
 1    China           _1_22_20           12
 2    China           _1_23_20           34
 [3   China           _1_24_20           54] <----- I want this row and all the next ones to appear as well

我的SQL查询：

WITH timestamped_table AS 
(
    SELECT
        Country_Region,
        [STRUCT<timestamp STRING, timestamp_value INT64>
        ('2020-2-22', _2_22_20),
        ('2020-2-23', _2_23_20)] AS timestamp_data
    FROM
        `username.bq_timeseries_covid19.recovered_global` 
    WHERE
        Country_Region = 'China' AND Province_State = 'Zhejiang'
)
SELECT
    Country_Region, timestamp
FROM
    timestamped_table
CROSS JOIN
    UNNEST(timestamped_table.timestamp_data) AS timestamp

现在唯一的问题是我必须从查询中扩展这些行：

STRUCT<timestamp STRING, timestamp_value INT64>
        ('2020-2-22', _2_22_20),
        ('2020-2-23', _2_23_20)

以便它们包含所有日期列

如果我通过一些脚本来扩展它，我会想象它是这样的，我认为这不是最好的主意：

STRUCT<timestamp STRING, timestamp_value INT64>
        for column in get_date_columns():
        (timestamp_from_columnname(), column)

---

我认为编写SQL脚本/函数会更好，但我不知道如何从这一点开始，你需要明确列出列。但您不需要CTE：

SELECT rg.Country_Region, el.timestamp, el.timestamp_value
FROM `username.bq_timeseries_covid19.recovered_global` rg CROSS JOIN
     UNNEST([STRUCT<timestamp STRING, timestamp_value INT64>
             ('2020-2-22', _2_22_20),
             ('2020-2-23', _2_23_20)
            ]) el
WHERE rg.Country_Region = 'China' and
      rg.Province_State = 'Zhejiang';

---

实际上，使用JSON和字符串操作有一些技巧，但在我看来，这些技巧非常粗糙，不值得付出努力。

您需要明确列出这些列。但您不需要CTE：

SELECT rg.Country_Region, el.timestamp, el.timestamp_value
FROM `username.bq_timeseries_covid19.recovered_global` rg CROSS JOIN
     UNNEST([STRUCT<timestamp STRING, timestamp_value INT64>
             ('2020-2-22', _2_22_20),
             ('2020-2-23', _2_23_20)
            ]) el
WHERE rg.Country_Region = 'China' and
      rg.Province_State = 'Zhejiang';

---

实际上，使用JSON和字符串操作有一些技巧，但在我看来，这些技巧非常粗糙，不值得付出任何努力。

下面是BigQuery标准SQL，不要求您显式列出列，无论您有多少列—50、50或150—它应该可以工作，而不需要对下面的代码进行任何更改

#standardSQL
SELECT id, Country_Region, 
  STRUCT(
      SPLIT(kv,':')[OFFSET(0)] AS ColumnName, 
      SPLIT(kv,':')[OFFSET(1)] AS Value
  ) AS `Date`
FROM `project.dataset.table` t,
UNNEST(SPLIT(REGEXP_REPLACE(TO_JSON_STRING(t), '[{}"]', ''))) kv
WHERE SPLIT(kv,':')[OFFSET(0)] NOT IN ('id', 'Country_Region')

您可以使用下面的简化示例中的虚拟数据测试、播放上述内容

#standardSQL
WITH `project.dataset.table` AS (
  SELECT 1 AS id, 'China' AS Country_Region, 12 AS _1_22_20, 34 AS _1_23_20, 54 AS _1_24_20 UNION ALL
  SELECT 2 AS id, 'Italy' AS Country_Region, 22 AS _1_22_20, 44 AS _1_23_20, 64 AS _1_24_20 
)
SELECT id, Country_Region, 
  STRUCT(
      SPLIT(kv,':')[OFFSET(0)] AS ColumnName, 
      SPLIT(kv,':')[OFFSET(1)] AS Value
  ) AS `Date`
FROM `project.dataset.table` t,
UNNEST(SPLIT(REGEXP_REPLACE(TO_JSON_STRING(t), '[{}"]', ''))) kv
WHERE SPLIT(kv,':')[OFFSET(0)] NOT IN ('id', 'Country_Region')

有输出

Row id  Country_Region  Date.ColumnName Date.Value   
1   1   China           _1_22_20        12   
2   1   China           _1_23_20        34   
3   1   China           _1_24_20        54   
4   2   Italy           _1_22_20        22   
5   2   Italy           _1_23_20        44   
6   2   Italy           _1_24_20        64   

---

下面是BigQuery标准SQL，不要求您显式列出列，无论您有多少列—50、50或150—它应该可以工作，而不需要对下面的代码进行任何更改

#standardSQL
SELECT id, Country_Region, 
  STRUCT(
      SPLIT(kv,':')[OFFSET(0)] AS ColumnName, 
      SPLIT(kv,':')[OFFSET(1)] AS Value
  ) AS `Date`
FROM `project.dataset.table` t,
UNNEST(SPLIT(REGEXP_REPLACE(TO_JSON_STRING(t), '[{}"]', ''))) kv
WHERE SPLIT(kv,':')[OFFSET(0)] NOT IN ('id', 'Country_Region')

您可以使用下面的简化示例中的虚拟数据测试、播放上述内容

#standardSQL
WITH `project.dataset.table` AS (
  SELECT 1 AS id, 'China' AS Country_Region, 12 AS _1_22_20, 34 AS _1_23_20, 54 AS _1_24_20 UNION ALL
  SELECT 2 AS id, 'Italy' AS Country_Region, 22 AS _1_22_20, 44 AS _1_23_20, 64 AS _1_24_20 
)
SELECT id, Country_Region, 
  STRUCT(
      SPLIT(kv,':')[OFFSET(0)] AS ColumnName, 
      SPLIT(kv,':')[OFFSET(1)] AS Value
  ) AS `Date`
FROM `project.dataset.table` t,
UNNEST(SPLIT(REGEXP_REPLACE(TO_JSON_STRING(t), '[{}"]', ''))) kv
WHERE SPLIT(kv,':')[OFFSET(0)] NOT IN ('id', 'Country_Region')

有输出

Row id  Country_Region  Date.ColumnName Date.Value   
1   1   China           _1_22_20        12   
2   1   China           _1_23_20        34   
3   1   China           _1_24_20        54   
4   2   Italy           _1_22_20        22   
5   2   Italy           _1_23_20        44   
6   2   Italy           _1_24_20        64   

---

您需要显式列出列。您需要显式列出列。