Dictionary 键，BigQuery中的值计数

BigQuery中以下问题的实现：

我有以下JSON格式的字典。如何计算id字典中键、值的总数？ {fil:{property:{id:{id_1:a，id_2:b，id_3:c，id_4:d}}

值a可以出现在多个这样的字典中的任何id_1、…、id_5中。需要计算a在任何字典的任何ID中出现的次数。 对于1，使用“显示选项”下的“使用旧版SQL”框取消选中，可以使用逗号运算符获取表和重复字段的叉积：

WITH MyTable AS (
  SELECT STRUCT(STRUCT(ARRAY<STRUCT<key STRING, value STRING>>[('id_1', 'a'), ('id_2', 'b'), ('id_3', 'c'), ('id_4', 'd')] AS id) AS property) AS fil
  UNION ALL SELECT STRUCT(STRUCT(ARRAY<STRUCT<key STRING, value STRING>>[('id_1', 'b'), ('id_3', 'e')] AS id) AS property) AS fil
  UNION ALL SELECT STRUCT(STRUCT(ARRAY<STRUCT<key STRING, value STRING>>[] AS id) AS property) AS fil
  UNION ALL SELECT STRUCT(STRUCT(ARRAY<STRUCT<key STRING, value STRING>>[('id_4', 'a'), ('id_2', 'c')] AS id) AS property) AS fil)
SELECT
  COUNT(DISTINCT id.key) AS num_keys,
  COUNT(DISTINCT id.value) AS num_values
FROM MyTable t, t.fil.property.id AS id;
+----------+------------+
| num_keys | num_values |
+----------+------------+
|        4 |          5 |
+----------+------------+

使用遗留SQL，您可以使用精确的计数来完成类似的事情，您可能不需要将其展平，尽管设置内联示例比较困难

---

对于2，您可以应用类似的方法，使用标准SQL展平，然后计算COUNTIFid.value=a的出现次数。或者，在传统SQL中，您可以使用COUNTt.fil.property.id.value=a.

假设您的字典以字符串形式存储在名为json的字段中

答案的关键是下面的查询

它解析json字段并提取所有键/值对及其父字典名

SELECT parent, key, value 
FROM JS((
  SELECT json FROM  
    (SELECT '{"fil":{"property":{"id":{"id_1":"a","id_2":"b","id_3":"c","id_4":"d"}}}}' AS json),
    (SELECT '{"fil":{"property":{"type":{"id_1":"x","id_2":"a","id_3":"y","id_4":"z"}, "category":{"id_1":"v","id_2":"w","id_3":"a","id_4":"b"}}}}' AS json)
  ),
  json,                                    // Input columns
  "[{name: 'parent', type:'string'},       // Output schema
   {name: 'key', type:'string'},
   {name: 'value', type:'string'}]",
   "function(r, emit) {                    // The function
      x = JSON.parse(r.json);
      processKey(x, '');
      function processKey(node, parent) {
        Object.keys(node).map(function(key) {
          value = node[key].toString();
          if (value !== '[object Object]') {
            emit({parent:parent, key:key, value:value});
          } else {
            if (parent !== '' && parent.substr(parent.length-1) !== '.') {parent += '.'};
            processKey(node[key], parent + key);
          };
        });         
      };
    }"
  )

以上查询结果如下

parent                  key     value    
fil.property.id         id_1    a    
fil.property.id         id_2    b    
fil.property.id         id_3    c    
fil.property.id         id_4    d    
fil.property.type       id_1    x    
fil.property.type       id_2    a    
fil.property.type       id_3    y    
fil.property.type       id_4    z    
fil.property.category   id_1    v    
fil.property.category   id_2    w    
fil.property.category   id_3    a    
fil.property.category   id_4    b

从那里，您可以很容易地得到两个答案：

Q1：如何计算每个字典中的键、值的总数

结果是

parent                  key_value_pairs  
fil.property.id         4    
fil.property.type       4    
fil.property.category   4    

问题2：需要计算任何值在任何字典的任何ID中出现的次数

---

由于其他答案对我来说很难，我制作了适用于string:int-dict的正则表达式

SELECT 
        *, REGEXP_EXTRACT_ALL(my_dict_column, r'"(\w+": \d+)') as keys
FROM test.test_table

---

从中，您可以执行键、值等操作

如何在GBQ表中存储词典。提供模式和示例很重要，因此，您可以使用投递答案左侧投票下方的勾号标记接受答案。看看为什么它很重要。同样重要的是对答案进行投票。投票选出有帮助的答案。还有更多。。。当有人回答你的问题时，你可以检查一下该做什么。

SELECT value, COUNT(1) AS value_appearances
FROM JS((
  SELECT json FROM  
    (SELECT '{"fil":{"property":{"id":{"id_1":"a","id_2":"b","id_3":"c","id_4":"d"}}}}' AS json),
    (SELECT '{"fil":{"property":{"type":{"id_1":"x","id_2":"a","id_3":"y","id_4":"z"}, "category":{"id_1":"v","id_2":"w","id_3":"a","id_4":"b"}}}}' AS json)
  ),
  json,                                    // Input columns
  "[{name: 'parent', type:'string'},       // Output schema
   {name: 'key', type:'string'},
   {name: 'value', type:'string'}]",
   "function(r, emit) {                    // The function
      x = JSON.parse(r.json);
      processKey(x, '');
      function processKey(node, parent) {
        Object.keys(node).map(function(key) {
          value = node[key].toString();
          if (value !== '[object Object]') {
            emit({parent:parent, key:key, value:value});
          } else {
            if (parent !== '' && parent.substr(parent.length-1) !== '.') {parent += '.'};
            processKey(node[key], parent + key);
          };
        });         
      };
    }"
  )
GROUP BY value  

value   value_appearances    
a       3    
b       2    
c       1    
d       1    
x       1    
y       1    
z       1    
v       1    
w       1    

SELECT 
        *, REGEXP_EXTRACT_ALL(my_dict_column, r'"(\w+": \d+)') as keys
FROM test.test_table