Java hadoop top10树映射没有输出任何数据
我有一段Java代码,它应该将10大树状图映射到Java hadoop top10树映射没有输出任何数据,java,hadoop,treemap,Java,Hadoop,Treemap,我有一段Java代码,它应该将10大树状图映射到hadoop。然而,当我运行代码时,我没有得到任何数据输出。我在哪里遗漏了什么 代码: 2018-07-10 09:16:19287信息[org.apache.hadoop.mapred.LocalJobRunner]-2018-07-10 09:16:19287 信息[org.apache.hadoop.mapred.MapTask]-开始刷新映射 输出2018-07-10 09:16:19288信息 [org.apache.hadoop.map
hadoop
。然而,当我运行代码时,我没有得到任何数据输出。我在哪里遗漏了什么
代码:
2018-07-10 09:16:19287信息[org.apache.hadoop.mapred.LocalJobRunner]-2018-07-10 09:16:19287
信息[org.apache.hadoop.mapred.MapTask]-开始刷新映射
输出2018-07-10 09:16:19288信息
[org.apache.hadoop.mapred.MapTask]-溢出地图输出2018-07-10
09:16:19288信息[org.apache.hadoop.mapred.MapTask]-bufstart=0;
bufend=10;bufvoid=104857600 2018-07-10 09:16:19291信息
[org.apache.hadoop.mapred.MapTask]-kvstart=26214396(104857584);
kvend=26214360(104857440);长度=37/6553600 2018-07-10
09:16:19307信息[org.apache.hadoop.mapred.MapTask]-开始刷新
地图输出时间2018-07-10 09:16:19307信息
[org.apache.hadoop.mapred.MapTask](重置)赤道0 kv
26214396(104857584)kvi 26214356(104857424)2018-07-10 09:16:19307
信息[org.apache.hadoop.mapred.MapTask]-溢出映射输出
2018-07-10 09:16:19307信息[org.apache.hadoop.mapred.MapTask]-
bufstart=0;bufend=10;bufvoid=104857600 2018-07-10
09:16:19727信息[org.apache.hadoop.mapred.MapTask]-kvstart=
26214396(104857584); kvend=26214360(104857440);长度=37/6553600
2018-07-10 09:16:19744信息[org.apache.hadoop.mapred.MapTask]-
在关闭期间忽略异常
org.apache.hadoop.mapred.MapTask$NewOutputCollector@61ccfc89
java.lang.ArrayIndexOutOfBoundsException:1位于
Reducer1_1.reduce(Reducer1_1.java:21)位于
com.zy.top10.Reducer1\u 1.reduce(Reducer1\u 1.java:1)
- 问题:System.out.println(“+text.toString());这个 text.toString()为空。为什么
public class Mapper1_1 extends Mapper<Object, Text, NullWritable, Text> {
public static final int limit = 10;
private TreeMap<Integer, Text> top10cats = new TreeMap<Integer, Text>();
private int flag=0;
private int tmp=0;
@Override
protected void map(Object key, Text value,
Mapper<Object, Text, NullWritable, Text>.Context context)
throws IOException, InterruptedException {
String line=value.toString();
if(!line.isEmpty()){
String[] words=line.split("\t");
if(flag!=5){
System.out.println(words[0]+"&&&&&&&&&&&&"+words[1]);
flag++;
}
top10cats.put(Integer.parseInt(words[1]), value);
if(top10cats.size()>limit){
tmp++;
top10cats.remove(top10cats.firstKey());
}
}
}
@Override
protected void cleanup(
Mapper<Object, Text, NullWritable, Text>.Context context)
throws IOException, InterruptedException {
System.out.println("&&&&&&&"+tmp);
for(Text text: top10cats.values()){
System.out.println("##########"+text.toString());
context.write(NullWritable.get(), text);
}
}
}
public class Reducer1_1 extends Reducer<NullWritable, Text, NullWritable, Text> {
public static final int N = 10;
private TreeMap<Integer, Text> top10cats = new TreeMap<Integer, Text>();
@Override
protected void reduce(NullWritable key, Iterable<Text> values,
Reducer<NullWritable, Text, NullWritable, Text>.Context context)
throws IOException, InterruptedException {
for (Text val : values) {
String tokens[] = val.toString().split("\t");
Integer weight = Integer.parseInt(tokens[1]);
top10cats.put(weight, val);
if (top10cats.size() > N)
top10cats.remove(top10cats.firstKey());
}
for(Text text:top10cats.values()){
context.write(NullWritable.get(), text);
}
}
}
2018-07-10 09:16:19,220 INFO [org.apache.hadoop.mapred.MapTask] -
mapreduce.task.io.sort.mb: 100 2018-07-10 09:16:19,220 INFO
[org.apache.hadoop.mapred.MapTask] - soft limit at 83886080
2018-07-10 09:16:19,220 INFO [org.apache.hadoop.mapred.MapTask] -
bufstart = 0; bufvoid = 104857600 2018-07-10 09:16:19,220 INFO
[org.apache.hadoop.mapred.MapTask] - kvstart = 26214396; length =
6553600 2018-07-10 09:16:19,226 INFO
[org.apache.hadoop.mapred.MapTask] - Map output collector class =
org.apache.hadoop.mapred.MapTask$MapOutputBuffer
1000473355&&&&&&&&&&&&39
1000473355 39
1001103582&&&&&&&&&&&&6
1001103582 6
1001698581&&&&&&&&&&&&2
1001698581 2
1004307115&&&&&&&&&&&&24
1004307115 24
1005179492&&&&&&&&&&&&2
1005179492 2
&&&&&&&1605
##########
##########
##########
##########
##########
##########
##########
##########
##########
##########