Java hadoop map reduce程序不显示输出
我有一个java的wriiten map reduce程序Java hadoop map reduce程序不显示输出,java,hadoop,mapreduce,hdfs,Java,Hadoop,Mapreduce,Hdfs,我有一个java的wriiten map reduce程序 public class myproject { public static class Map extends Mapper<LongWritable, Text, Text, Text> { private final static IntWritable one = new IntWritable(1); private Text word = new Text(); publi
public class myproject {
public static class Map extends Mapper<LongWritable, Text, Text, Text> {
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
public void map(LongWritable key, Text value, OutputCollector<Text, Text> output, Reporter reporter) throws IOException {
String line = value.toString();
/*
MY CODE
*/
output.collect(new Text(string1),new Text(string2));
}
}
public class Reduce extends Reducer<Text, Text, Text, IntWritable> {
public void reduce(Text key, Iterator<Text> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
while (values.hasNext()) {
/*
MY CODE
*/
values.next();
}
output.collect(key, new IntWritable(val));
}
}
public static void main(String[] args) throws Exception {
JobConf conf = new JobConf(myproject.class);
conf.setJobName("myprojectwork");
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(IntWritable.class);
conf.setMapOutputKeyClass(Text.class);
conf.setMapOutputValueClass(Text.class);
conf.setMapperClass(Map.class);
conf.setCombinerClass(Reduce.class);
conf.setReducerClass(Reduce.class);
conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(TextOutputFormat.class);
FileInputFormat.setInputPaths(conf, new Path(args[0]));
FileOutputFormat.setOutputPath(conf, new Path(args[1]));
JobClient.runJob(conf);
它只显示这个,然后退出
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
当我尝试列出项目文件夹/输出文件夹时
它不显示这样的文件/目录
wordcount问题工作得非常好,但我的程序不工作
有什么我遗漏的吗
我是否必须编辑hadoop目录中的任何.xml文件或类似文件?您不是在混合使用2个hadoop api吗?在较新的api中,上下文对象不是收集器。将映射方法更改为公共void映射(LongWritable键、文本值、上下文上下文)引发IOException、InterruptedException。reducer toook也是一样,我现在更改了它,它在线程“main”java.io.IOException中显示以下错误异常:作业失败!在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)在sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)在myproject.main(myproject.java:68)在org.apache.apache.apache在org.apache.hadoop.util.RunJar.main(RunJar.java:212)的java.lang.reflect.Method.invoke(Method.java:483)中的异常部分,我已经声明在map和reduce函数中都显示IOException。请编辑代码并发布运行的内容好吗?
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable