Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/hadoop/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
在hadoop中写入多个文件夹?_Hadoop - Fatal编程技术网

在hadoop中写入多个文件夹?

在hadoop中写入多个文件夹?,hadoop,Hadoop,我正在尝试将我的输出从reducer分离到不同的文件夹 My dirver has the following code: FileOutputFormat.setOutputPath(job, new Path(output)); //MultipleOutputs.addNamedOutput(job, namedOutput, outputFormatClass, keyClass, valueClass) //MultipleOutput

我正在尝试将我的输出从reducer分离到不同的文件夹

My dirver has the following code:
 FileOutputFormat.setOutputPath(job, new Path(output));
            //MultipleOutputs.addNamedOutput(job, namedOutput, outputFormatClass, keyClass, valueClass)
            //MultipleOutputs.addNamedOutput(job, namedOutput, outputFormatClass, keyClass, valueClass)
            MultipleOutputs.addNamedOutput(job, "foo", TextOutputFormat.class, NullWritable.class, Text.class);
            MultipleOutputs.addNamedOutput(job, "bar", TextOutputFormat.class, Text.class,NullWritable.class);
            MultipleOutputs.addNamedOutput(job, "foobar", TextOutputFormat.class, Text.class, NullWritable.class);

And then my reducer has the following code:
mos.write("foo",NullWritable.get(),new Text(jsn.toString()));
mos.write("bar", key,NullWritable.get());
mos.write("foobar", key,NullWritable.get());

But in the output, I see:

output/foo-r-0001
output/foo-r-0002
output/foobar-r-0001
output/bar-r-0001


But what I am trying is :

output/foo/part-r-0001
output/foo/part-r-0002
output/bar/part-r-0001
输出/foobar/part-r-0001

我该怎么做? 谢谢

如果你是这个意思,最简单的方法就是你自己做以下事情之一:--

  • 使用带基本输出路径的命名输出
  • 没有命名输出且仅使用基本输出路径
  • 在您的情况下,这是第1点,因此,请更改以下内容--

    对,

    其中,“foo/part”、“bar/part”和“foobar/part”对应于baseOutputPath。 因此,将创建foo、bar和foobar目录,并在part-r-xxxxx文件中创建这些目录

    您也可以尝试上面的第2点,它实际上不需要任何命名的输出


    如果需要进一步澄清,请联系我。

    这是什么版本的Hadoop?
    mos.write("foo",NullWritable.get(),new Text(jsn.toString()));
    mos.write("bar", key,NullWritable.get());
    mos.write("foobar", key,NullWritable.get());
    
    mos.write("foo",NullWritable.get(),new Text(jsn.toString()), "foo/part");
    mos.write("bar", key,NullWritable.get(), "bar/part");
    mos.write("foobar", key,NullWritable.get(), "foobar/part");