Java 向hadoop提交远程应用程序时如何设置jar路径?
我尝试从at网站运行第一个示例 下面是我的代码Java 向hadoop提交远程应用程序时如何设置jar路径?,java,hadoop,yarn,Java,Hadoop,Yarn,我尝试从at网站运行第一个示例 下面是我的代码 Path jarPath = new Path("target/HadoopStudy-0.0.1-SNAPSHOT.jar"); FileSystem fs = FileSystem.get(conf); FileStatus jarStatus = fs.getFileStatus(jarPath); LocalResource amJarRsrc = Records.newRecord(LocalResour
Path jarPath = new Path("target/HadoopStudy-0.0.1-SNAPSHOT.jar");
FileSystem fs = FileSystem.get(conf);
FileStatus jarStatus = fs.getFileStatus(jarPath);
LocalResource amJarRsrc = Records.newRecord(LocalResource.class);
amJarRsrc.setType(LocalResourceType.FILE);
amJarRsrc.setVisibility(LocalResourceVisibility.APPLICATION);
logger.debug(ConverterUtils.getYarnUrlFromPath(jarPath));
amJarRsrc.setResource(ConverterUtils.getYarnUrlFromPath(jarPath));
amJarRsrc.setTimestamp(jarStatus.getModificationTime());
amJarRsrc.setSize(jarStatus.getLen());
localResources.put(Common.LOCAL_JAR_PATH, amJarRsrc);
amContainer.setLocalResources(localResources);
但我可以获得应用程序id,并将应用程序提交给yarn,但应用程序始终会失败,我在master上检查了NodeManager日志,发现了这些错误
java.net.URISyntaxException: Relative path in absolute URI: ://target/HadoopStudy-0.0.1-SNAPSHOT.jar
at java.net.URI.checkPath(URI.java:1804)
at java.net.URI.<init>(URI.java:752)
at org.apache.hadoop.yarn.util.ConverterUtils.getPathFromYarnURL(ConverterUtils.java:77)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourceRequest.<init>(LocalResourceRequest.java:46)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:523)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourcesTransition.transition(ContainerImpl.java:494)
at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:850)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:73)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:681)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:674)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:134)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:81)
at java.lang.Thread.run(Thread.java:722)
java.net.URISyntaxException:绝对URI中的相对路径://target/HadoopStudy-0.0.1-SNAPSHOT.jar
位于java.net.URI.checkPath(URI.java:1804)
位于java.net.URI。(URI.java:752)
位于org.apache.hadoop.warn.util.ConverterUtils.getPathFromyarnur(ConverterUtils.java:77)
位于org.apache.hadoop.warn.server.nodemanager.containermanager.localizer.LocalResourceRequest.(LocalResourceRequest.java:46)
位于org.apache.hadoop.warn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourceTransition.transition(ContainerImpl.java:523)
位于org.apache.hadoop.warn.server.nodemanager.containermanager.container.ContainerImpl$RequestResourceTransition.transition(ContainerImpl.java:494)
位于org.apache.hadoop.warn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
位于org.apache.hadoop.warn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
位于org.apache.hadoop.warn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
位于org.apache.hadoop.warn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
位于org.apache.hadoop.warn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:850)
位于org.apache.hadoop.warn.server.nodemanager.containermanager.container.ContainerImpl.handle(ContainerImpl.java:73)
位于org.apache.hadoop.warn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:681)
位于org.apache.hadoop.warn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher.handle(ContainerManagerImpl.java:674)
位于org.apache.hadoop.warn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:134)
位于org.apache.hadoop.warn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:81)
运行(Thread.java:722)
我在pca上运行这个示例,但我在pcb上部署hadoop,jar在pca上
经过调查,我无法修复它。但我有一些问题。1) 我需要将jar上传到hdfs吗?如果需要,我知道该怎么做
2) 我只是想了解一些纱线的原理,如果我想提交申请,我需要把罐子上传给师傅吗?我可以将jar文件上传到master的本地文件系统吗?如果可以,如何设置路径
将HadoopStudy-0.0.1-SNAPSHOT.jar路径更改为完整路径我可以使用相对路径吗?我实际上不知道。但这是我从堆栈跟踪中了解到的。