Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark 本地路径FileNotFound上的Apache Beam WriteToText_Apache Spark_Apache Beam_Apache Beam Io - Fatal编程技术网

Apache spark 本地路径FileNotFound上的Apache Beam WriteToText

Apache spark 本地路径FileNotFound上的Apache Beam WriteToText,apache-spark,apache-beam,apache-beam-io,Apache Spark,Apache Beam,Apache Beam Io,我试图通过Docker运行的Spark作业来运行Apache Beam作业 以下是管道: pipeline\u options=PipelineOptions([“--runner=PortableRunner”, “--job_endpoint=localhost:8099”], 管道类型(检查=真) 输入='kinglear.txt' #将beam.Pipeline(选项=PipelineOptions(管道类型检查=True))作为p: 将beam.Pipeline(选项=Pipeline

我试图通过Docker运行的Spark作业来运行Apache Beam作业

以下是管道:

pipeline\u options=PipelineOptions([“--runner=PortableRunner”,
“--job_endpoint=localhost:8099”],
管道类型(检查=真)
输入='kinglear.txt'
#将beam.Pipeline(选项=PipelineOptions(管道类型检查=True))作为p:
将beam.Pipeline(选项=Pipeline_选项)设为p:
#将文本文件[pattern]读入PCollection。
def输出(x):
打印(str(x))
返回x
行=(p
|beam.io.ReadFromText(输入))
#计算每个单词出现的次数。
计数=(
线
|“Split'>>(beam.FlatMap(lambda x:re.findall(r'[A-Za-z\']+',x))
.具有_输出_类型(unicode))
|“成对体”>>束图(λx:(x,1))
|'GroupAndSum'>>beam.CombinePerKey(总和)
|'WriteToText'>>WriteToText(os.path.join(os.getcwd(),'/output/part'))
管道运行良好,直到最后一步,我得到以下信息:

RuntimeError: FileNotFoundError: [Errno 2] No such file or directory: '/output/beam-temp-part-5039fdd2d6c511e9b2ed2c4d54e984b7/a0ac2e38-dd8d-4748-8700-f1a114de0e1d.part.gz' [while running 'WriteToText/Write/WriteImpl/FinalizeWrite']
我应该如何指定输出路径