Sql 从值由管道分隔的列中删除重复项

数据如下所示-

category_id   category_name    associated_keys
    111          Books         CC34DE5W|SQA7ZZ87|LM24NO3P|SQA7ZZ87
    222          Office        LM24NO3P|AAB12B34
    444         Furniture      X34YY78Z|LM24NO3P|SQA7ZZ87|SEF5C6T4|CC34DE5W|AAB12B34
    222          Office        X34YY78Z|X34YY78Z

category_id   category_name    associated_keys
    111          Books         CC34DE5W|SQA7ZZ87|LM24NO3P
    222          Office        LM24NO3P|AAB12B34
    444         Furniture      X34YY78Z|LM24NO3P|SQA7ZZ87|SEF5C6T4|CC34DE5W|AAB12B34
    222          Office        X34YY78Z

我想从关联的\u keys列中删除不同类别\u id的重复项-

category_id   category_name    associated_keys
    111          Books         CC34DE5W|SQA7ZZ87|LM24NO3P|SQA7ZZ87
    222          Office        LM24NO3P|AAB12B34
    444         Furniture      X34YY78Z|LM24NO3P|SQA7ZZ87|SEF5C6T4|CC34DE5W|AAB12B34
    222          Office        X34YY78Z|X34YY78Z

category_id   category_name    associated_keys
    111          Books         CC34DE5W|SQA7ZZ87|LM24NO3P
    222          Office        LM24NO3P|AAB12B34
    444         Furniture      X34YY78Z|LM24NO3P|SQA7ZZ87|SEF5C6T4|CC34DE5W|AAB12B34
    222          Office        X34YY78Z

---

不要将这些值存储为字符串！BigQuery提供数组。因此，我将向您展示如何将字符串的结果转换为数组：

with t as (
      select 111 as category_id, 'Books' as category_name, 'CC34DE5W|SQA7ZZ87|LM24NO3P|SQA7ZZ87' as associated_keys union all
      select 222, 'Office', 'LM24NO3P|AAB12B34' union all
      select 444, 'Furniture', 'X34YY78Z|LM24NO3P|SQA7ZZ87|SEF5C6T4|CC34DE5W|AAB12B34' union all
      select 222, 'Office', 'X34YY78Z|X34YY78Z'
     )
select t.* except (associated_keys),
       (select array_agg(distinct key)
        from unnest(split(t.associated_keys, '|')) key
       ) as associated_keys
from t;

---

如果您确实想重建字符串，可以使用

string\u agg（）

，但我不建议这样做。

不要将此类值存储为字符串！BigQuery提供数组。因此，我将向您展示如何将字符串的结果转换为数组：

with t as (
      select 111 as category_id, 'Books' as category_name, 'CC34DE5W|SQA7ZZ87|LM24NO3P|SQA7ZZ87' as associated_keys union all
      select 222, 'Office', 'LM24NO3P|AAB12B34' union all
      select 444, 'Furniture', 'X34YY78Z|LM24NO3P|SQA7ZZ87|SEF5C6T4|CC34DE5W|AAB12B34' union all
      select 222, 'Office', 'X34YY78Z|X34YY78Z'
     )
select t.* except (associated_keys),
       (select array_agg(distinct key)
        from unnest(split(t.associated_keys, '|')) key
       ) as associated_keys
from t;

---

如果您确实想重建字符串，可以使用

string\u agg（）

#standardSQL
SELECT category_id, category_name, 
  (SELECT STRING_AGG(DISTINCT key, '|') FROM UNNEST(SPLIT(associated_keys, '|')) key) associated_keys
FROM (
  SELECT category_id, category_name, STRING_AGG(associated_keys, '|') AS associated_keys
  FROM `project.dataset.data` 
  GROUP BY category_id, category_name  
)   

如果要应用于示例中的样本数据，则输出为

Row category_id category_name   associated_keys  
1   111         Books           CC34DE5W|SQA7ZZ87|LM24NO3P   
2   222         Office          LM24NO3P|AAB12B34|X34YY78Z   
3   444         Furniture       X34YY78Z|LM24NO3P|SQA7ZZ87|SEF5C6T4|CC34DE5W|AAB12B34       

如果您不想按类别\u id进行分组（如上一个问题中所示），请使用下面的选项

#standardSQL
SELECT category_id, category_name, 
  (SELECT STRING_AGG(DISTINCT key, '|') FROM UNNEST(SPLIT(associated_keys, '|')) key) associated_keys
FROM `project.dataset.data` 

有输出

Row category_id category_name   associated_keys  
1   111         Books           CC34DE5W|SQA7ZZ87|LM24NO3P   
2   222         Office          LM24NO3P|AAB12B34    
3   444         Furniture       X34YY78Z|LM24NO3P|SQA7ZZ87|SEF5C6T4|CC34DE5W|AAB12B34    
4   222         Office          X34YY78Z

下面是BigQuery标准SQL

#standardSQL
SELECT category_id, category_name, 
  (SELECT STRING_AGG(DISTINCT key, '|') FROM UNNEST(SPLIT(associated_keys, '|')) key) associated_keys
FROM (
  SELECT category_id, category_name, STRING_AGG(associated_keys, '|') AS associated_keys
  FROM `project.dataset.data` 
  GROUP BY category_id, category_name  
)   

如果要应用于示例中的样本数据，则输出为

Row category_id category_name   associated_keys  
1   111         Books           CC34DE5W|SQA7ZZ87|LM24NO3P   
2   222         Office          LM24NO3P|AAB12B34|X34YY78Z   
3   444         Furniture       X34YY78Z|LM24NO3P|SQA7ZZ87|SEF5C6T4|CC34DE5W|AAB12B34       

如果您不想按类别\u id进行分组（如上一个问题中所示），请使用下面的选项

#standardSQL
SELECT category_id, category_name, 
  (SELECT STRING_AGG(DISTINCT key, '|') FROM UNNEST(SPLIT(associated_keys, '|')) key) associated_keys
FROM `project.dataset.data` 

有输出

Row category_id category_name   associated_keys  
1   111         Books           CC34DE5W|SQA7ZZ87|LM24NO3P   
2   222         Office          LM24NO3P|AAB12B34    
3   444         Furniture       X34YY78Z|LM24NO3P|SQA7ZZ87|SEF5C6T4|CC34DE5W|AAB12B34    
4   222         Office          X34YY78Z