Google cloud dataflow 运行WordAccount示例数据流运行程序

Google cloud dataflow 运行WordAccount示例数据流运行程序,google-cloud-dataflow,apache-beam,word-count,Google Cloud Dataflow,Apache Beam,Word Count,我将在DataFlow Runner上执行我的Wordcount项目,我使用beam文档中的命令,但我得到了以下错误: C:\BIGDATA_FORMATION\TP_WC_GCP\word-count-beam>mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args="--runner=DataflowRunner --gcpTempLocation=gs://b

我将在DataFlow Runner上执行我的Wordcount项目,我使用beam文档中的命令,但我得到了以下错误:

C:\BIGDATA_FORMATION\TP_WC_GCP\word-count-beam>mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args="--runner=DataflowRunner --gcpTempLocation=gs://bucket_wc/tmp/ --project=WordCount --inputFile=gs://bucket_wc/test_file.txt --output=gs://bucket_wc/counts" -Pdataflow-runner
[INFO] Scanning for projects...
[INFO]
[INFO] --------------------< org.example:word-count-beam >---------------------
[INFO] Building word-count-beam 0.1
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ word-count-beam ---
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\BIGDATA_FORMATION\TP_WC_GCP\word-count-beam\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ word-count-beam ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ word-count-beam ---
sept. 01, 2020 5:16:16 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
INFOS: No stagingLocation provided, falling back to gcpTempLocation
[WARNING]
java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
    at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod (InstanceBuilder.java:224)
    at org.apache.beam.sdk.util.InstanceBuilder.build (InstanceBuilder.java:155)
    at org.apache.beam.sdk.PipelineRunner.fromOptions (PipelineRunner.java:55)
    at org.apache.beam.sdk.Pipeline.create (Pipeline.java:147)
    at org.apache.beam.examples.WordCount.runWordCount (WordCount.java:176)
    at org.apache.beam.examples.WordCount.main (WordCount.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
    at java.lang.Thread.run (Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod (InstanceBuilder.java:214)
    at org.apache.beam.sdk.util.InstanceBuilder.build (InstanceBuilder.java:155)
    at org.apache.beam.sdk.PipelineRunner.fromOptions (PipelineRunner.java:55)
    at org.apache.beam.sdk.Pipeline.create (Pipeline.java:147)
    at org.apache.beam.examples.WordCount.runWordCount (WordCount.java:176)
    at org.apache.beam.examples.WordCount.main (WordCount.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
    at java.lang.Thread.run (Thread.java:745)
Caused by: java.lang.IllegalArgumentException: No files to stage has been found.
    at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions (DataflowRunner.java:281)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod (InstanceBuilder.java:214)
    at org.apache.beam.sdk.util.InstanceBuilder.build (InstanceBuilder.java:155)
    at org.apache.beam.sdk.PipelineRunner.fromOptions (PipelineRunner.java:55)
    at org.apache.beam.sdk.Pipeline.create (Pipeline.java:147)
    at org.apache.beam.examples.WordCount.runWordCount (WordCount.java:176)
    at org.apache.beam.examples.WordCount.main (WordCount.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
    at java.lang.Thread.run (Thread.java:745)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  7.202 s
[INFO] Finished at: 2020-09-01T17:16:17+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project word-count-beam: An exception occured while executing the Java class. Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions): InvocationTargetException: No files to stage has been found. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
C:\BIGDATA\u FORMATION\TP\u WC\u GCP\word count beam>mvn compile exec:java-Dexec.mainClass=org.apache.beam.examples.WordCount-Dexec.args=“--runner=DataflowRunner--gcpTempLocation=gs://bucket\u WC/tmp/--project=WordCount--inputFile gs://bucket\u WC/test\u file.txt--output=gs://bucket\u WC counts/counts”-Pdataflow runner
[信息]正在扫描项目。。。
[信息]
[信息]-----------------------------------
[信息]建筑字数梁0.1
[信息]------------------------------------[jar]---------------------------------
[信息]
[信息]---maven资源插件:2.6:resources(默认资源)@word count beam---
[警告]使用平台编码(实际上是Cp1252)复制过滤后的资源,即构建依赖于平台!
[信息]跳过不存在的资源目录C:\BIGDATA\u FORMATION\TP\u WC\u GCP\word count beam\src\main\resources
[信息]
[信息]---maven编译器插件:3.7.0:compile(默认编译)@word count beam---
[信息]无需编译-所有类都是最新的
[信息]
[信息]---exec maven插件:1.6.0:java(默认cli)@word count beam---
2020年9月1日下午5:16:16 org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create
信息:未提供stagingLocation,退回到GCP位置
[警告]
java.lang.RuntimeException:无法从工厂方法DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)构造实例
位于org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
位于org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
位于org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
位于org.apache.beam.sdk.Pipeline.create(Pipeline.java:147)
位于org.apache.beam.examples.WordCount.runWordCount(WordCount.java:176)
位于org.apache.beam.examples.WordCount.main(WordCount.java:192)
位于sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)
位于sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)处
位于java.lang.reflect.Method.invoke(Method.java:498)
位于org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
在java.lang.Thread.run(Thread.java:745)
原因:java.lang.reflect.InvocationTargetException
位于sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)
位于sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)处
位于java.lang.reflect.Method.invoke(Method.java:498)
位于org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
位于org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
位于org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
位于org.apache.beam.sdk.Pipeline.create(Pipeline.java:147)
位于org.apache.beam.examples.WordCount.runWordCount(WordCount.java:176)
位于org.apache.beam.examples.WordCount.main(WordCount.java:192)
位于sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)
位于sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)处
位于java.lang.reflect.Method.invoke(Method.java:498)
位于org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
在java.lang.Thread.run(Thread.java:745)
原因:java.lang.IllegalArgumentException:未找到要暂存的文件。
位于org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:281)
位于sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)
位于sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)处
位于java.lang.reflect.Method.invoke(Method.java:498)
位于org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
位于org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
位于org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
位于org.apache.beam.sdk.Pipeline.create(Pipeline.java:147)
位于org.apache.beam.examples.WordCount.runWordCount(WordCount.java:176)
位于org.apache.beam.examples.WordCount.main(WordCount.java:192)
位于sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)
位于sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)处
位于java.lang.reflect.Method.invoke(Method.java:498)
位于org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
在java.lang.Thread.run(Thread.java:745)
[信息]------------------------------------------------------------------------
[信息]生成失败
[信息]------------------------------------------------------------------------
[信息]总时间:7.202秒
[信息]完成时间:2020-09-01T17:16:17+01:00
[信息]------------------------------------------------------------------------
[错误]无法在project word count beam上执行目标org.codehaus.mojo:exec maven plugin:1.6.0:java(默认cli):执行java类时发生异常。未能从工厂方法DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)构造实例:InvocationTargetException:未找到要暂存的文件。->[帮助1]
[错误]
[错误]要查看错误的完整堆栈跟踪,请使用-e开关重新运行Maven。
[错误]使用-X开关重新运行Maven以启用完整调试日志记录。
[错误]
[呃
mvn archetype:generate \
      -DarchetypeGroupId=org.apache.beam \
      -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
      -DarchetypeVersion=2.23.0 \
      -DgroupId=org.example \
      -DartifactId=word-count-beam \
      -Dversion="0.1" \
      -Dpackage=org.apache.beam.examples \
      -DinteractiveMode=false
mvn -Pdataflow-runner compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args="--project=<PROJECT_ID> \
      --gcpTempLocation=gs://<BUCKET_NAME>/tmp/ \
      --stagingLocation=gs://<BUCKET_NAME>/staging/ \
      --output=gs://<BUCKET_NAME>/output \
      --runner=DataflowRunner \
      --inputFile=gs://<BUCKET_NAME>/test.txt \
      --region=us-west1"