Java 如何修复src文件系统问题上的资源更改
我试图在MR上使用Java 如何修复src文件系统问题上的资源更改,java,hadoop,hive,bigdata,Java,Hadoop,Hive,Bigdata,我试图在MR上使用Hive执行SQL,但中途失败,错误如下: Application application_1570514228864_0001 failed 2 times due to AM Container for appattempt_1570514228864_0001_000002 exited with exitCode: -1000 Failing this attempt.Diagnostics: [2019-10-08 13:57:49.272]Failed to dow
Hive
执行SQL
,但中途失败,错误如下:
Application application_1570514228864_0001 failed 2 times due to AM Container for appattempt_1570514228864_0001_000002 exited with exitCode: -1000
Failing this attempt.Diagnostics: [2019-10-08 13:57:49.272]Failed to download resource { { s3a://tpcds/tmp/hadoop-yarn/staging/root/.staging/job_1570514228864_0001/libjars, 1570514262820, FILE, null },pending,[(container_1570514228864_0001_02_000001)],1132444167207544,DOWNLOADING} java.io.IOException: Resource s3a://tpcds/tmp/hadoop-yarn/staging/root/.staging/job_1570514228864_0001/libjars changed on src filesystem (expected 1570514262820, was 1570514269265
从我的角度来看,错误日志中的关键信息是在src文件系统上更改了libjars(预期为1570514262820,为1570514269265
。目前有几个关于此问题的线程,但尚未得到回答,如和
我从和中发现了一些有价值的东西。我通过NTP
通过所有相关节点同步了时钟。但同样的问题仍然存在
欢迎发表任何评论,谢谢。我仍然不知道为什么资源文件的时间戳不一致,并且没有办法通过配置方式修复它,好的 然而,我设法找到了一个解决方法来跳过这个问题。让我在这里为可能遇到相同问题的任何人总结一下 通过检查错误日志并在
Hadoop
源代码中搜索,我们可以在Hadoop-thread项目/Hadoop-thread/Hadoop-thread-common/src/main/java/org/apache/Hadoop/thread/util/FSDownload.java
中跟踪问题
只需删除异常抛出语句
private void verifyAndCopy(Path destination)
throws IOException, YarnException {
final Path sCopy;
try {
sCopy = resource.getResource().toPath();
} catch (URISyntaxException e) {
throw new IOException("Invalid resource", e);
}
FileSystem sourceFs = sCopy.getFileSystem(conf);
FileStatus sStat = sourceFs.getFileStatus(sCopy);
if (sStat.getModificationTime() != resource.getTimestamp()) {
/**
throw new IOException("Resource " + sCopy +
" changed on src filesystem (expected " + resource.getTimestamp() +
", was " + sStat.getModificationTime());
**/
LOG.debug("[Gearon][Info] The timestamp is not consistent among resource files.\n" +
"Stop throwing exception . It doesn't affect other modules. ");
}
if (resource.getVisibility() == LocalResourceVisibility.PUBLIC) {
if (!isPublic(sourceFs, sCopy, sStat, statCache)) {
throw new IOException("Resource " + sCopy +
" is not publicly accessible and as such cannot be part of the" +
" public cache.");
}
}
downloadAndUnpack(sCopy, destination);
}
构建hadoop纱线项目并将“hadoop-warn-common-x.x.x.jar复制到
$hadoop\u HOME/share/hadoop/warn”
将此线程留在这里,感谢您对如何在不更改hadoop源代码的情况下修复此线程的进一步解释。我也必须这样做,这应该是可配置的,即使很小的延迟也会导致执行失败,这可能会发生,如果您将hadoop文件系统更改为使用s3并运行MR程序,请注意*请确保正在使用相同的jdk版本为您的hadoop版本生成apache hadoop文档中提到的jar,否则您可能会遇到错误。osx 10.15.4上的相同问题,类似于
resource.getTimestamp()
drop毫秒,其中FileStatus
保持不变