在hadoop中写入多个文件夹?
我正在尝试将我的输出从reducer分离到不同的文件夹在hadoop中写入多个文件夹?,hadoop,Hadoop,我正在尝试将我的输出从reducer分离到不同的文件夹 My dirver has the following code: FileOutputFormat.setOutputPath(job, new Path(output)); //MultipleOutputs.addNamedOutput(job, namedOutput, outputFormatClass, keyClass, valueClass) //MultipleOutput
My dirver has the following code:
FileOutputFormat.setOutputPath(job, new Path(output));
//MultipleOutputs.addNamedOutput(job, namedOutput, outputFormatClass, keyClass, valueClass)
//MultipleOutputs.addNamedOutput(job, namedOutput, outputFormatClass, keyClass, valueClass)
MultipleOutputs.addNamedOutput(job, "foo", TextOutputFormat.class, NullWritable.class, Text.class);
MultipleOutputs.addNamedOutput(job, "bar", TextOutputFormat.class, Text.class,NullWritable.class);
MultipleOutputs.addNamedOutput(job, "foobar", TextOutputFormat.class, Text.class, NullWritable.class);
And then my reducer has the following code:
mos.write("foo",NullWritable.get(),new Text(jsn.toString()));
mos.write("bar", key,NullWritable.get());
mos.write("foobar", key,NullWritable.get());
But in the output, I see:
output/foo-r-0001
output/foo-r-0002
output/foobar-r-0001
output/bar-r-0001
But what I am trying is :
output/foo/part-r-0001
output/foo/part-r-0002
output/bar/part-r-0001
输出/foobar/part-r-0001
我该怎么做?
谢谢如果你是这个意思,最简单的方法就是你自己做以下事情之一:--
如果需要进一步澄清,请联系我。这是什么版本的Hadoop?
mos.write("foo",NullWritable.get(),new Text(jsn.toString()));
mos.write("bar", key,NullWritable.get());
mos.write("foobar", key,NullWritable.get());
mos.write("foo",NullWritable.get(),new Text(jsn.toString()), "foo/part");
mos.write("bar", key,NullWritable.get(), "bar/part");
mos.write("foobar", key,NullWritable.get(), "foobar/part");