Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/java/309.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/hadoop/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java Hadoop Mapreduce MultipleInputs无法加载映射器类_Java_Hadoop_Mapreduce_Yarn - Fatal编程技术网

Java Hadoop Mapreduce MultipleInputs无法加载映射器类

Java Hadoop Mapreduce MultipleInputs无法加载映射器类,java,hadoop,mapreduce,yarn,Java,Hadoop,Mapreduce,Yarn,我正在我们的纱线集群上使用新的MapReduce Api。我需要从两个不同的目录中读取两种不同格式的文件。为此,我决定使用多个输入来指定这两个映射器类。下面是我的工作司机 Job job = new Job(new Configuration(), "Daily Report"); job.setJarByClass(MyDailyJob.class); MultipleInputs.addInputPath(job, new Path(args[0]), TextInputFormat.c

我正在我们的纱线集群上使用新的MapReduce Api。我需要从两个不同的目录中读取两种不同格式的文件。为此,我决定使用多个输入来指定这两个映射器类。下面是我的工作司机

Job job = new Job(new Configuration(), "Daily Report");

job.setJarByClass(MyDailyJob.class);

MultipleInputs.addInputPath(job, new Path(args[0]), TextInputFormat.class, Record1ParsingMapper.class);
MultipleInputs.addInputPath(job, new Path(args[1]), TextInputFormat.class, Record2ParsingMapper.class);


FileOutputFormat.setOutputPath(job, new Path(args[2]));

job.setReducerClass(ReportParsingReducer.class);
job.setNumReduceTasks(10);

job.setOutputKeyClass(Text.class);
job.setOutputValueClass(NullWritable.class);

return (job.waitForCompletion(true) ? 0 : 1); 
我的映射器具有以下定义: 公共类Record1ParsingMapper扩展了Mapper

运行此作业时,出现以下异常:

Exception in thread "main" java.lang.NoClassDefFoundError: org/json/simple/parser/ParseException
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1986)
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1951)
    at org.apache.hadoop.mapreduce.lib.input.MultipleInputs.getMapperTypeMap(MultipleInputs.java:141)
    at org.apache.hadoop.mapreduce.lib.input.DelegatingInputFormat.getSplits(DelegatingInputFormat.java:60)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:498)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:515)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:399)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1313)
    at main.java.com.adnear.mr.jobs.MyDailyJob.run(MyDailyJob.java:70)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    at main.java.com.adnear.mr.jobs.MyDailyJob.main(MyDailyJob.java:226)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

Caused by: java.lang.ClassNotFoundException: org.json.simple.parser.ParseException\
在下面的语句中,它似乎在配置类的getClassByName()方法中失败。 clazz=Class.forName(name,true,classLoader)

我正在正确指定映射器类的路径。有人能解释一下是什么导致了这个类加载异常吗

谢谢, 发展

错误消息明确指出类路径中缺少某些类。此类是运行时执行
作业所需的
json-simple-1.1.1.jar
jar文件的一部分。因此,将此jar文件添加到类路径中以成功运行作业。 您可以从获取jar文件。

希望有帮助

可能是您的
记录1 ParsingMapper
是一个内部类?如果找到答案,您可以关闭该问题吗?如果问题解决了,请张贴您的解决方案。
Caused by: java.lang.ClassNotFoundException: org.json.simple.parser.ParseException