Google cloud dataflow 轮询结果文件超时：启动Google Dataflow SQL查询时出错

我对Google数据流和Apache Beam还不熟悉。我正在尝试通过DataflowSQL部署dataflow作业。作业从pubsub获取输入并写入bigquery

当我使用Dataflow SQL启动作业时，我在轮询结果文件中得到错误超时。 此错误既适用于gcloud CLI，也适用于Google云控制台

下面是错误的片段

2020-05-16 21:46:50.193 ISTTimeout in polling result file: gs://dataflow-staging-us-central1-XXXXXXXXXXX/staging/template_launches/2020-05-16_09_06_52-11619869258908426171/operaton_result. Possible causes are: 1. Your launch takes too long time to finish. Please check the logs on stackdriver. 2. Service account XXXXXXXXXXXX-compute@developer.gserviceaccount.com may not have enough permissions to pull container image gcr.io/dataflow-sql-templates/dataflow-sql-launcher-templates/dataflow_sql_launcher_image:20200303-rc00 or create new objects in gs://dataflow-staging-us-central1-XXXXXXXXXXXXX/staging/template_launches/2020-05-16_09_06_52-11619869258908426171/operaton_result. 3. Transient errors occurred, please try again. 

我想这个日志中提到的原因不可能是真的，因为我很少能成功地部署作业

对于成功的案例，数据流SQL查询中SELECT和FROM之间的参数小于10。随着SELECT和FROM之间的参数数量增加到18，出现了这个问题

下面是我正在使用的一个示例查询

        SELECT MIN(sensor.event_date) as day,
            MIN(sensor.event_timestamp) as event_timestamp,
            TUMBLE_START("INTERVAL 1 MINUTE") as parameters_timestamp,  
            sensor.sensor_id as sensor_id,
            AVG(CAST(sensor.param2 AS FLOAT64)) as param2,
            AVG(CAST(sensor.param3 AS FLOAT64)) as param3,
            AVG(CAST(sensor.param4 AS FLOAT64)) as param4,
            AVG(CAST(sensor.param5 AS FLOAT64)) as param5,
            AVG(CAST(sensor.param6 AS FLOAT64)) as param6,
            AVG(CAST(sensor.param7 AS FLOAT64)) as param7,
            AVG(CAST(sensor.param8 AS FLOAT64)) as param8,
            AVG(CAST(sensor.param9 AS FLOAT64)) as param9,
            AVG(CAST(sensor.param10 AS FLOAT64)) as param10,
            AVG(CAST(sensor.param11 AS FLOAT64)) as param11,
            AVG(CAST(sensor.param12 AS FLOAT64)) as param12,
            AVG(CAST(sensor.param13 AS FLOAT64)) as param13,
            AVG(CAST(sensor.param14 AS FLOAT64)) as param14,
            AVG(CAST(sensor.param15 AS FLOAT64)) as param15
    FROM pubsub.topic.`project1`.`table1` AS sensor
    GROUP BY
      sensor.sensor_id,
      TUMBLE(sensor.event_timestamp, "INTERVAL 1 MINUTE")

发生错误时，我发现存储路径中没有控制台日志（即2020-05-16_09_06_52-11619869258908426171/操作结果）。 gs://dataflow-staging-us-central1-xxxxxxxxxx/staging/template\u启动/

谢谢， 贾根

这里还有作业日志的图像。

---

您之前提到您已成功运行此作业。你在工作中有什么改变吗？我建议联系谷歌云支持。如果不深入了解作业执行的细节，就很难调试它。我看到其他一些用户也出现了类似的错误，因为容器映像的构建不正确，所以请确保您正确地执行了以下步骤：@rmestives是的，我将dataflow SQL查询中SELECT和FROM之间的参数数增加到了18。我现在将参数数减少到9。在这个缩减之后，我能够启动DataflowSQL作业。我可以在SELECT查询中输入的参数数量是否有限制？@chamikara我想问题在于SELECT查询中参数或表达式的数量。我观察到9或小于9的表达式，我能够部署作业。使用10个或更多表达式时，作业模板本身不会启动。SELECT Dataflow SQL查询中的表达式数量是否有限制。@JaganKumarG我在文档中看不到任何与限制相关的内容。您可以分享您的查询或部分查询吗？