Google cloud dataflow Google数据流工作流错误_Google Cloud Dataflow

Google cloud dataflow Google数据流工作流错误

google-cloud-dataflow

Google cloud dataflow Google数据流工作流错误,google-cloud-dataflow,Google Cloud Dataflow,我对同一项目中的存储桶进行了批读写操作。我在尝试写入输出时看到此异常。有什么想法吗 (c1a5d1aff2d8459b): java.lang.RuntimeException: com.google.cloud.dataflow.sdk.util.UserCodeException: java.io.IOException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request {

我对同一项目中的存储桶进行了批读写操作。我在尝试写入输出时看到此异常。有什么想法吗

(c1a5d1aff2d8459b): java.lang.RuntimeException: com.google.cloud.dataflow.sdk.util.UserCodeException: java.io.IOException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request { "code" : 400, "errors" : [ { "domain" : "global", "message" : "No object name", "reason" : "required" } ], "message" : "No object name" } at com.google.cloud.dataflow.sdk.runners.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:160) at com.google.cloud.dataflow.sdk.util.DoFnRunnerBase$DoFnContext.outputWindowedValue(DoFnRunnerBase.java:288) at com.google.cloud.dataflow.sdk.util.DoFnRunnerBase$DoFnContext.outputWindowedValue(DoFnRunnerBase.java:284) at com.google.cloud.dataflow.sdk.util.DoFnRunnerBase$DoFnProcessContext$1.outputWindowedValue(DoFnRunnerBase.java:508) at com.google.cloud.dataflow.sdk.util.GroupAlsoByWindowsViaIteratorsDoFn.processElement(GroupAlsoByWindowsViaIteratorsDoFn.java:123) at com.google.cloud.dataflow.sdk.util.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:49) at com.google.cloud.dataflow.sdk.util.DoFnRunnerBase.processElement(DoFnRunnerBase.java:139) at com.google.cloud.dataflow.sdk.runners.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:188) at com.google.cloud.dataflow.sdk.runners.worker.ForwardingParDoFn.processElement(ForwardingParDoFn.java:42) at com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerLoggingParDoFn.processElement(DataflowWorkerLoggingParDoFn.java:47) at com.google.cloud.dataflow.sdk.util.common.worker.ParDoOperation.process(ParDoOperation.java:55) at com.google.cloud.dataflow.sdk.util.common.worker.OutputReceiver.process(OutputReceiver.java:52) at com.google.cloud.dataflow.sdk.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:221) at com.google.cloud.dataflow.sdk.util.common.worker.ReadOperation.start(ReadOperation.java:182) at com.google.cloud.dataflow.sdk.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:69) at com.google.cloud.dataflow.sdk.runners.worker.DataflowWorker.executeWork(DataflowWorker.java:284) at com.google.cloud.dataflow.sdk.runners.worker.DataflowWorker.doWork(DataflowWorker.java:220) at com.google.cloud.dataflow.sdk.runners.worker.DataflowWorker.getAndPerformWork(DataflowWorker.java:170) at com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerHarness$WorkerThread.doWork(DataflowWorkerHarness.java:192) at com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerHarness$WorkerThread.call(DataflowWorkerHarness.java:172) at com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerHarness$WorkerThread.call(DataflowWorkerHarness.java:159) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) （c1a5d1aff2d8459b）：java.lang.RuntimeException:com.google.cloud.dataflow.sdk.util.UserCodeException:java.io.IOException:com.google.api.client.googleapis.json.GoogleJsonResponseException:400错误请求 { “代码”：400， “错误”：[{ “域”：“全局”， “消息”：“无对象名称”， “原因”：“必需” } ], “消息”：“无对象名称” } 位于com.google.cloud.dataflow.sdk.runners.worker.SimpleParDoFn$1.output（SimpleParDoFn.java:160）位于com.google.cloud.dataflow.sdk.util.DoFnRunnerBase$DoFnContext.outputWindowedValue（DoFnRunnerBase.java:288）位于com.google.cloud.dataflow.sdk.util.DoFnRunnerBase$DoFnContext.outputWindowedValue（DoFnRunnerBase.java:284）位于com.google.cloud.dataflow.sdk.util.DoFnRunnerBase$DoFnProcessContext$1.outputWindowedValue（DoFnRunnerBase.java:508）位于com.google.cloud.dataflow.sdk.util.groupalsobywindowsviateratorsdofn.processElement（groupalsobywindowsviateratorsdofn.java:123）位于com.google.cloud.dataflow.sdk.util.SimpleDofRunner.invokeProcessElement（SimpleDofRunner.java:49）位于com.google.cloud.dataflow.sdk.util.DoFnRunnerBase.processElement（DoFnRunnerBase.java:139）位于com.google.cloud.dataflow.sdk.runners.worker.SimpleParDoFn.processElement（SimpleParDoFn.java:188）位于com.google.cloud.dataflow.sdk.runners.worker.ForwardingParDoFn.processElement（ForwardingParDoFn.java:42）位于com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerLoggingParDoFn.processElement（DataflowWorkerLoggingParDoFn.java:47）位于com.google.cloud.dataflow.sdk.util.common.worker.ParDoOperation.process（ParDoOperation.java:55）位于com.google.cloud.dataflow.sdk.util.common.worker.OutputReceiver.process（OutputReceiver.java:52）位于com.google.cloud.dataflow.sdk.util.common.worker.ReadOperation.runReadLoop（ReadOperation.java:221）在com.google.cloud.dataflow.sdk.util.common.worker.ReadOperation.start（ReadOperation.java:182）上位于com.google.cloud.dataflow.sdk.util.common.worker.MapTaskExecutor.execute（MapTaskExecutor.java:69）位于com.google.cloud.dataflow.sdk.runners.worker.DataflowWorker.executeWork（DataflowWorker.java:284）位于com.google.cloud.dataflow.sdk.runners.worker.DataflowWorker.doWork（DataflowWorker.java:220）位于com.google.cloud.dataflow.sdk.runners.worker.DataflowWorker.getAndPerformWork（DataflowWorker.java:170）位于com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerHarness$WorkerThread.doWork（DataflowWorkerHarness.java:192）在com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerHarness$WorkerThread.call（DataflowWorkerHarness.java:172）上在com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerHarness$WorkerThread.call（DataflowWorkerHarness.java:159）上在java.util.concurrent.FutureTask.run（FutureTask.java:266）处位于java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1142）位于java.util.concurrent.ThreadPoolExecutor$Worker.run（ThreadPoolExecutor.java:617）运行（Thread.java:745）

这是由于为

TextIO.Write

指定了不正确的路径造成的（缺少GCS bucket-一个示例正确路径是

gs://some bucket/some output prefix

，而在此作业中，它被简单地指定为

gs://some output prefix

）

这应该在管道施工时，在工人开始工作之前被发现。这是Apache Beam和Dataflow SDK验证GCS路径时的一个错误。我正在修复，请按照PR进行更新jkff 10分钟前

这是由于为

TextIO.Write

指定了不正确的路径造成的（缺少GCS bucket-一个示例正确路径是

gs://some bucket/some output prefix

，而在此作业中，它被指定为简单的

gs://some output prefix

）

这应该在管道施工时，在工人开始工作之前被发现。这是Apache Beam和Dataflow SDK验证GCS路径时的一个错误。我正在修复，请按照PR进行更新jkff 10分钟前

什么是工作ID？“2017-04-19_10_45_04-17412713462337636976”什么是工作ID？“2017-04-19_10_45_04-17412713462337636976”