Google cloud dataflow apache_beam.runners.dataflow_runner.DataflowRuntimeException:数据流管道失败:

Google cloud dataflow apache_beam.runners.dataflow_runner.DataflowRuntimeException:数据流管道失败:,google-cloud-dataflow,google-cloud-ml,apache-beam,Google Cloud Dataflow,Google Cloud Ml,Apache Beam,我在CloudShell中设置了一个GoogleCloud项目,并尝试运行此教程脚本 遇到以下错误: ***@***:~/git/cloudml-samples/flowers$ ./sample.sh Your active configuration is: [cloudshell-4691] Using job id: flowers_***_20170113_162148 python trainer/preprocess.py \ --input_dict "$DICT_

我在CloudShell中设置了一个GoogleCloud项目,并尝试运行此教程脚本

遇到以下错误:

***@***:~/git/cloudml-samples/flowers$ ./sample.sh 
Your active configuration is: [cloudshell-4691]


Using job id:  flowers_***_20170113_162148

python trainer/preprocess.py \
  --input_dict "$DICT_FILE" \
  --input_path "gs://cloud-ml-data/img/flower_photos/eval_set.csv" \
  --output_path "${GCS_PATH}/preproc/eval" \
  --cloud
WARNING:root:Using fallback coder for typehint: Any.
WARNING:root:Using fallback coder for typehint: Any.
WARNING:root:Using fallback coder for typehint: Any.
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting google-cloud-dataflow==0.4.4
  Using cached google-cloud-dataflow-0.4.4.zip
  Saved /tmp/tmpSoHiTi/google-cloud-dataflow-0.4.4.zip
Successfully downloaded google-cloud-dataflow
# Takes about 30 mins to preprocess everything.  We serialize the two
Traceback (most recent call last):
  File "trainer/preprocess.py", line 436, in <module>
    main(sys.argv[1:])
  File "trainer/preprocess.py", line 432, in main
    run(arg_dict)
  File "trainer/preprocess.py", line 353, in run
    p.run()
  File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 159, in run
    return self.runner.run(self)
  File "/home/slalomconsultingsf/.local/lib/python2.7/site-packages/apache_beam/runners/dataflow_runner.py", line 195, in run
    % getattr(self, 'last_error_msg', None), self.result)
apache_beam.runners.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed:
(b85b0a598a3565cb): Workflow failed.
***@***:~/git/cloudml-samples/flowers$。/sample.sh
您的活动配置是:[cloudshell-4691]
使用作业id:flowers\u***\ u 20170113\u 162148
python trainer/preprocess.py\
--输入dict“$dict文件”\
--输入路径“gs://cloud ml data/img/flower\u photos/eval\u set.csv”\
--输出路径“${GCS\u path}/preproc/eval”\
--云彩
警告:root:对typehint:Any使用回退编码器。
警告:root:对typehint:Any使用回退编码器。
警告:root:对typehint:Any使用回退编码器。
弃用:pip安装--下载已弃用,将在将来删除。Pip现在有一个应该使用的下载命令。
正在收集google云数据流==0.4.4
使用缓存的google-cloud-dataflow-0.4.4.zip
已保存/tmp/tmpSoHiTi/google-cloud-dataflow-0.4.4.zip
已成功下载google云数据流
#预处理所有内容大约需要30分钟。我们将二者连载
回溯(最近一次呼叫最后一次):
文件“trainer/preprocess.py”,第436行,在
main(sys.argv[1:])
文件“trainer/preprocess.py”,第432行,主目录
运行(arg_dict)
文件“trainer/preprocess.py”,第353行,运行中
p、 运行()
文件“/home/slalomconsultingsf/.local/lib/python2.7/site packages/apache_beam/pipeline.py”,第159行,正在运行
返回self.runner.run(self)
文件“/home/slalomconsultingsf/.local/lib/python2.7/site packages/apache_beam/runners/dataflow_runner.py”,第195行,运行中
%getattr(self,'last\u error\u msg',None),self.result)
apache_beam.runners.dataflow_runner.DataflowRuntimeException:数据流管道失败:
(b85b0a598a3565cb):工作流失败。
我无法从哪里知道我做错了什么


感谢您对疑难解答的解答和帮助。

启用数据流API。在万神殿顶部的搜索框中,键入“数据流api”将带您进入一个窗口,您可以单击“启用api”


我想这会帮你解决的。我禁用了数据流API,得到了与您相同的错误,然后当重新启用时,问题又消失了。

刚刚测试过,这是正确的修复方法。作业正在运行。谢谢:-)