Java 运行ExportSnapshot时引发ClassNotFoundException
当我尝试从HBase主节点运行ExportSnapshot时,我遇到了一个令人困惑的ClassNotFoundExceptionJava 运行ExportSnapshot时引发ClassNotFoundException,java,mapreduce,hbase,Java,Mapreduce,Hbase,当我尝试从HBase主节点运行ExportSnapshot时,我遇到了一个令人困惑的ClassNotFoundExceptionhbase shell和其他命令工作正常,我的集群完全可以运行 这感觉像是一个类路径问题,但我不知道我遗漏了什么 $ /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot ambarismoketest-snapshot -copy-to hdfs://10.0.1.90/app
hbase shell
和其他命令工作正常,我的集群完全可以运行
这感觉像是一个类路径问题,但我不知道我遗漏了什么
$ /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot ambarismoketest-snapshot -copy-to hdfs://10.0.1.90/apps/hbase/data -mappers 16
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2015-10-13 20:05:02,339 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
2015-10-13 20:05:04,351 INFO [main] util.FSVisitor: No logs under directory:hdfs://cinco-de-nameservice/apps/hbase/data/.hbase-snapshot/impression_event_production_hbase-transfer-to-staging-20151013/WALs
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/Job
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.runCopyJob(ExportSnapshot.java:529)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:646)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:697)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:701)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.Job
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 5 more
问题 这是因为mapreduce2 JAR在类路径中不可用。类路径设置正确,但我没有在该节点上安装mapreduce2客户端。HBase的ExportSnapshot显然在将快照导出到另一个集群时依赖于这些客户端JAR,因为它会写入HDFS 修复 如果您使用Ambari:
ClassNotFoundException,当ExportSnapshot在hadoop集群上运行时抛出,其中HBase与resourcemanager未安装在同一节点上。标题暗示安装资源管理器是解决方案,这可能会起作用;然而,关键是您需要类路径中的hadoop mapreduce2 JAR,只需安装mapreduce2客户机即可
具体来说,对于我们来说,快照导出前一天正常工作,第二天又出现故障的原因是我们的HBase主服务器打开了另一个问题的b/c。我们的备份HBase主机没有mapreduce2客户机JAR,但原始主主机有