Google bigquery 大查询视图已经开始打破过去的工作模式,重复字段不再工作
我有很多观点,都是在这种基本结构下形成的Google bigquery 大查询视图已经开始打破过去的工作模式,重复字段不再工作,google-bigquery,Google Bigquery,我有很多观点,都是在这种基本结构下形成的 { "bar": "barvalue", "foo":[ { "nested": "value" }, { "nested":"value2"} } 因此,我有如下的工作观点 select bar as bar, foo.nested as nested flatten([table name],foo) 但在更复杂的视图中,这周已停止工作。由于旧错误,无法保存视图。无法为字段成员创建有效的输
{
"bar": "barvalue",
"foo":[
{ "nested": "value" },
{ "nested":"value2"}
}
因此,我有如下的工作观点
select
bar as bar,
foo.nested as nested
flatten([table name],foo)
但在更复杂的视图中,这周已停止工作。由于旧错误,无法保存视图。无法为字段成员创建有效的输出架构。将nested重命名为foo.nested,我使用flatte解析它。还有人看到这个吗?有什么解决办法吗?我已经加载了与失败的语法完全相同的视图,因此非常确定这是一个新的功能错误
所以我认为一个特定的值现在失败了,它是这样的
Last modified Schema Total Rows Total Bytes Expiration Labels
----------------- ------------------------------------------ ------------ ------------- ------------ --------
06 Sep 22:23:45 |- description: string 4 1450
|- adminCreated: boolean
+- members: record (repeated)
| |- status: string
| |- kind: string
| |- email: string
| |- etag: string
| |- role: string
| |- type: string
| |- id: string
+- scanner: record
| |- scanversion: float
|- id: string
+- aliases: record (repeated)
| |- alias: string
+- nonEditableAliases: record (repeated)
| |- alias: string
|- kind: string
|- name: string
|- directMembersCount: string
|- etag: string
|- email: string
还有一个
Last modified Schema Total Rows Total Bytes Expiration Labels
----------------- ----------------------------------- ------------ ------------- ------------ --------
06 Sep 22:23:46 |- displayName: string 11 20361
|- name: string
|- organizationId: string
|- creationTime: string
|- lifecycleState: string
+- owner: record
| |- directoryCustomerId: string
+- bindings: record (repeated)
| |- role: string
| +- members: record (repeated)
| | |- member: string
+- scanner: record
| |- scanversion: float
然后,像这样的视图由于错误而失败
SELECT
"groups/"+email AS name,
g.members.email AS member,
g.members.role AS role,
REGEXP_EXTRACT(g.members.email,r'.*@(.*)') AS domain
FROM
FLATTEN([<projectid>:<dataset>.group],members) g
JOIN
[<projectid>:<dataset>.organization] o
ON
o.scanner.scanversion = g.scanner.scanversion
好的,在进一步的跟踪和错误之后,我找到了一个解决方法,可以将视图更改为这个工作
SELECT
violation,
name,
parent,
members.role as role,
members.member as member
FROM (
SELECT
"Invalid group member not in %orgdescription% domains" AS violation,
name,
parent,
role AS members.role,
member AS members.member
FROM (
SELECT
"groups/"+g.email AS name,
o.name AS parent,
g.members.role AS role,
g.members.email AS member,
REGEXP_EXTRACT(g.members.email,r'.*@(.*)') AS domain
FROM
FLATTEN([<projectid>:<dataset>.group],members) g
JOIN
[<projectid>:<dataset>.organizationhd] o
ON
o.scanner.scanversion = g.scanner.scanversion
WHERE
g.members.email IS NOT NULL)
WHERE
domain NOT IN('foo.com'))
为我工作!成功地将视图保存为“选择栏为栏”,foo.nested为“从展平[table_name]嵌套”,foo将细化示例,因为在44个视图中,只有1个视图失败,我这样做的原因比我想象的更微妙。但是,对于我已经工作了几个月的视图来说,这种情况刚刚开始发生。在您问题的底部,该视图的错误消息到底是怎么说的?作为一项建议,我建议您开始使用bigquery而不是legacy。这是一些旧代码,无法重构,这是突然出现的。这些观点已经奏效好几个月了。