Google cloud platform 无法从数据流SQL访问BigQuery
我正在尝试将pubsub中的一些数据与bigquery表连接起来Google cloud platform 无法从数据流SQL访问BigQuery,google-cloud-platform,google-bigquery,google-cloud-dataflow,google-cloud-pubsub,Google Cloud Platform,Google Bigquery,Google Cloud Dataflow,Google Cloud Pubsub,我正在尝试将pubsub中的一些数据与bigquery表连接起来 select st.*, wo.workorder from pubsub.topic.newcobi.sensor_ticks as st left join bigquery.table.newcobi.masterdata.workorders as wo on st.machine_id = wo.machine 这应该与google教程上的示例非常相似。但它总是在以下方面失败: Error messag
select
st.*,
wo.workorder
from pubsub.topic.newcobi.sensor_ticks as st
left join
bigquery.table.newcobi.masterdata.workorders as wo on st.machine_id = wo.machine
这应该与google教程上的示例非常相似。但它总是在以下方面失败:
Error message from worker: java.io.IOException: Failed to start reading from source: org.apache.beam.runners.core.construction.UnboundedReadFromBoundedSource$BoundedToUnboundedSourceAdapter@6283e583
org.apache.beam.runners.dataflow.worker.WorkerCustomSources$UnboundedReaderIterator.start(WorkerCustomSources.java:792)
(...)
Caused by: java.lang.UnsupportedOperationException: BigQuery storage source must be split before reading
org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageSourceBase.createReader(BigQueryStorageSourceBase.java:169)
org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageTableSource.createReader(BigQueryStorageTableSource.java:43)
(...)
无论我将结果写入bigquery还是pubsub,都会发生这种情况。那么:在阅读之前,我如何拆分表格
谢谢
Stefan只是确认一下-您是从Dataflow SQL UI执行此操作的吗?另外,您的BQ表的大小是多少?这个问题有任何更新吗?只是确认一下-您是从Dataflow SQL UI执行此操作的吗?还有,你的BQ表有多大?这个问题有更新吗?