Hadoop 由于缺少jar,无法配置hive.exec挂钩
我正在尝试使用配置单元,并使用“use db”命令切换数据库。我的设置是Hadoop 2.4.0和Hive 0.13.1。我将以下3个属性添加到.settings文件中Hadoop 由于缺少jar,无法配置hive.exec挂钩,hadoop,hive,yarn,Hadoop,Hive,Yarn,我正在尝试使用配置单元,并使用“use db”命令切换数据库。我的设置是Hadoop 2.4.0和Hive 0.13.1。我将以下3个属性添加到.settings文件中 set hive.exec.failure.hooks=org.apache.hadoop.hive.ql.hooks.ATSHook; set hive.exec.post.hooks=org.apache.hadoop.hive.ql.hooks.ATSHook; set hive.exec.pre.hooks=org.ap
set hive.exec.failure.hooks=org.apache.hadoop.hive.ql.hooks.ATSHook;
set hive.exec.post.hooks=org.apache.hadoop.hive.ql.hooks.ATSHook;
set hive.exec.pre.hooks=org.apache.hadoop.hive.ql.hooks.ATSHook;
然后打开配置单元命令行,通过“hive-I my.settings”传入.settings文件,然后获得:
hive> use db;
hive.exec.pre.hooks Class not found:org.apache.hadoop.hive.ql.hooks.ATSHook
FAILED: Hive Internal Error: java.lang.ClassNotFoundException(org.apache.hadoop.hive.ql.hooks.ATSHook)
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
我的类路径中似乎缺少一个jar。我尝试在web上搜索一个包含“org.apache.hadoop.hive.ql.hooks.atshake”类的jar,但没有成功。我尝试通过以下方式将所有包含JAR的路径从HIVE_HOME添加到Thread-site.xml:
<property>
<name>yarn.application.classpath</name>
<value>
...
/apps/hive/hive-0.13.1/hcatalog/share/hcatalog/*,
/apps/hive/hive-0.13.1/hcatalog/share/hcatalog/storage-handlers/hbase/lib/*,
/apps/hive/hive-0.13.1/hcatalog/share/webhcat/java-client/*,
/apps/hive/hive-0.13.1/hcatalog/share/webhcat/svr/lib/*,
/apps/hive/hive-0.13.1/lib/*
</value>
</property>
.application.classpath
...
/apps/hive/hive-0.13.1/hcatalog/share/hcatalog/*,
/apps/hive/hive-0.13.1/hcatalog/share/hcatalog/storage handlers/hbase/lib/*,
/apps/hive/hive-0.13.1/hcatalog/share/webhcat/java-client/*,
/apps/hive/hive-0.13.1/hcatalog/share/webhcat/svr/lib/*,
/apps/hive/hive-0.13.1/lib/*
还是不走运。有人知道我还需要做些什么来配置这些属性吗?显然,jar只在尚未发布的Hive 0.14.0中可用。因此,我不得不下载并根据网站上的说明构建蜂巢。这很简单:
mvn clean install -DskipTests -Phadoop-2
一旦建成,我就能做到这一点:
hive> add jar <HIVE_HOME>/ql/target
> ;
hive>添加jar/ql/target
> ;
或者将此属性添加到hive-site.xml
<property>
<name>hive.aux.jars.path</name>
<value>file:///<HIVE_HOME>/ql/target/hive-exec-0.14.0-SNAPSHOT.jar</value>
</property>
hive.aux.jars.path
file:////ql/target/hive-exec-0.14.0-SNAPSHOT.jar
我还发现了一个很好的幻灯片分享演示