Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/java/368.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/hadoop/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java 是否可以更改驱动程序类中使用的vCore数?_Java_Hadoop_Mapreduce_Yarn - Fatal编程技术网

Java 是否可以更改驱动程序类中使用的vCore数?

Java 是否可以更改驱动程序类中使用的vCore数?,java,hadoop,mapreduce,yarn,Java,Hadoop,Mapreduce,Yarn,我提交了一份关于纱线的MR作业,它只使用2个V孔,因此速度很慢 我已经设置了conf.set(“mapreduce.map.cpu.vcores”,“35”)在我的驱动程序代码中,即 Configuration conf = new Configuration(); GenericOptionsParser optionParser = new GenericOptionsParser(conf, args); String[] remainingArgs = optionParser.getR

我提交了一份关于纱线的MR作业,它只使用2个V孔,因此速度很慢

我已经设置了
conf.set(“mapreduce.map.cpu.vcores”,“35”)在我的驱动程序代码中,即

Configuration conf = new Configuration();
GenericOptionsParser optionParser = new GenericOptionsParser(conf, args);
String[] remainingArgs = optionParser.getRemainingArgs();
if (!(remainingArgs.length != 2 || remainingArgs.length != 4)) {
    System.err.println("Usage: DataProcessorMR <in> <out> <config> <daten-tool>");
    System.exit(2);
}
Job job = Job.getInstance(conf, "DataProcessorMR");
job.setJarByClass(DataProcessorMR.class);
job.setMapperClass(DPMapper.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(NullWritable.class);
job.addCacheFile(new Path(args[2]).toUri());
job.addCacheFile(new Path(args[3]).toUri());
conf.set("mapreduce.map.cpu.vcores", "35");
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
System.exit(job.waitForCompletion(true) ? 0 : 1);
Configuration conf=new Configuration();
GenericOptions Parser optionParser=新的GenericOptions Parser(conf,args);
String[]remainingArgs=optionParser.getRemainingArgs();
如果(!(remainargs.length!=2 | | remainargs.length!=4)){
System.err.println(“用法:DataProcessorMR”);
系统出口(2);
}
Job Job=Job.getInstance(conf,“DataProcessorMR”);
setJarByClass(DataProcessorMR.class);
setMapperClass(DPMapper.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(NullWritable.class);
job.addCacheFile(新路径(args[2]).toUri());
job.addCacheFile(新路径(args[3]).toUri());
conf.set(“mapreduce.map.cpu.vcores”,“35”);
addInputPath(作业,新路径(args[0]);
setOutputPath(作业,新路径(args[1]);
系统退出(作业等待完成(真)?0:1;

但这对纱线资源管理器没有影响?如何设置java类中MapReduce作业的vCore数。

conf.set(“MapReduce.map.cpu.VCORES”,“35”);应该在Job=Job.getInstance(conf,“DataProcessorMR”)之前设置;它是否提高了你的工作绩效??我对此表示怀疑,但实际上并非如此。它仍然使用2个内核。您有什么建议吗?我必须为此编写一个纱线应用程序吗?在capacity-scheduler.xml集合->纱线.调度器.产能.资源计算器=org.apache.hadoop.warn.util.resource.DominantResourceCalculator中,在此之后,您将在调度器选项卡中看到纱线UI上使用的70芯。