Hive Hcatalog蜂巢问题_Hive_Hcatalog

Hive Hcatalog蜂巢问题

hive

Hive Hcatalog蜂巢问题,hive,hcatalog,Hive,Hcatalog,我正试图从以下链接执行此hcatalog示例：运行作业时，我遇到以下异常 Exception in thread "main" com.google.common.util.concurrent.ExecutionError: java.lang.NoClassDefFoundError: org/antlr/runtime/RecognitionException at com.google.common.cache.LocalCache$Segment.get(LocalCach

我正试图从以下链接执行此

hcatalog

示例：

运行作业时，我遇到以下异常

Exception in thread "main" com.google.common.util.concurrent.ExecutionError: java.lang.NoClassDefFoundError: org/antlr/runtime/RecognitionException
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2232)

    at com.google.common.cache.LocalCache.get(LocalCache.java:3965)

    at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)

    at org.apache.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:167)

    at org.apache.hcatalog.common.HiveClientCache.get(HiveClientCache.java:143)

    at org.apache.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:544)

    at org.apache.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:103)

    at org.apache.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:85)

    at org.apache.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:85)

    at org.apache.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:54)

    at org.apache.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:46)

    at com.otsi.hcat.UseHCat.run(UseHCat.java:69)

    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

    at com.otsi.hcat.UseHCat.main(UseHCat.java:96)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:606)

    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

Caused by: java.lang.NoClassDefFoundError: org/antlr/runtime/RecognitionException
    at java.lang.Class.forName0(Native Method)

    at java.lang.Class.forName(Class.java:270)

    at org.apache.hadoop.hive.metastore.MetaStoreUtils.getClass(MetaStoreUtils.java:1378)

    at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:64)

    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:498)

    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:476)

    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:524)

    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:398)

    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:357)

    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:54)

    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:59)

    at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4948)

    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:171)

    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:154)

    at org.apache.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveClientCache.java:246)

    at org.apache.hcatalog.common.HiveClientCache$4.call(HiveClientCache.java:170)

    at org.apache.hcatalog.common.HiveClientCache$4.call(HiveClientCache.java:167)

    at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)

    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)

    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)

    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)

    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)

    ... 19 more

Caused by: java.lang.ClassNotFoundException: org.antlr.runtime.RecognitionException

线程“main”中的异常com.google.common.util.concurrent.ExecutionError:java.lang.NoClassDefFoundError:org/antlr/runtime/RecognitionException 位于com.google.common.cache.LocalCache$Segment.get（LocalCache.java:2232）位于com.google.common.cache.LocalCache.get（LocalCache.java:3965）位于com.google.common.cache.LocalCache$LocalManualCache.get（LocalCache.java:4764）位于org.apache.hcatalog.common.HiveClientCache.getOrCreate（HiveClientCache.java:167）位于org.apache.hcatalog.common.HiveClientCache.get（HiveClientCache.java:143）在org.apache.hcatalog.common.HCatUtil.getHiveClient（HCatUtil.java:544）上位于org.apache.hcatalog.mapreduce.InitializeInput.getInputJobInfo（InitializeInput.java:103）位于org.apache.hcatalog.mapreduce.InitializeInput.setInput（InitializeInput.java:85）位于org.apache.hcatalog.mapreduce.HCatInputFormat.setInput（HCatInputFormat.java:85）位于org.apache.hcatalog.mapreduce.HCatInputFormat.setInput（HCatInputFormat.java:54）位于org.apache.hcatalog.mapreduce.HCatInputFormat.setInput（HCatInputFormat.java:46）在com.otsi.hcat.UseHCat.run（UseHCat.java:69）位于org.apache.hadoop.util.ToolRunner.run（ToolRunner.java:70）位于org.apache.hadoop.util.ToolRunner.run（ToolRunner.java:84）位于com.otsi.hcat.UseHCat.main（UseHCat.java:96）在sun.reflect.NativeMethodAccessorImpl.invoke0（本机方法）处在sun.reflect.NativeMethodAccessorImpl.invoke（NativeMethodAccessorImpl.java:57）中在sun.reflect.DelegatingMethodAccessorImpl.invoke（DelegatingMethodAccessorImpl.java:43）中位于java.lang.reflect.Method.invoke（Method.java:606）位于org.apache.hadoop.util.RunJar.main（RunJar.java:212）原因：java.lang.NoClassDefFoundError:org/antlr/runtime/RecognitionException 位于java.lang.Class.forName0（本机方法）位于java.lang.Class.forName（Class.java:270）位于org.apache.hadoop.hive.metastore.MetaStoreUtils.getClass（MetaStoreUtils.java:1378）位于org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy（RawStoreProxy.java:64）位于org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore（HiveMetaStore.java:498）位于org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS（HiveMetaStore.java:476）位于org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB（HiveMetaStore.java:524）位于org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init（HiveMetaStore.java:398）位于org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler。（HiveMetaStore.java:357）位于org.apache.hadoop.hive.metastore.RetryingHMSHandler（RetryingHMSHandler.java:54）位于org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy（RetryingHMSHandler.java:59）位于org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler（HiveMetaStore.java:4948）位于org.apache.hadoop.hive.metastore.HiveMetaStoreClient。（HiveMetaStoreClient.java:171）位于org.apache.hadoop.hive.metastore.HiveMetaStoreClient。（HiveMetaStoreClient.java:154）位于org.apache.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient。（HiveClientCache.java:246）位于org.apache.hcatalog.common.HiveClientCache$4.call（HiveClientCache.java:170）位于org.apache.hcatalog.common.HiveClientCache$4.call（HiveClientCache.java:167）位于com.google.common.cache.LocalCache$LocalManualCache$1.load（LocalCache.java:4767）位于com.google.common.cache.LocalCache$LoadingValueReference.loadFuture（LocalCache.java:3568）位于com.google.common.cache.LocalCache$Segment.loadSync（LocalCache.java:2350）位于com.google.common.cache.LocalCache$Segment.lockedGetOrLoad（LocalCache.java:2313）位于com.google.common.cache.LocalCache$Segment.get（LocalCache.java:2228） ... 还有19个原因：java.lang.ClassNotFoundException:org.antlr.runtime.RecognitionException 在运行

MR作业之前

我执行了以下命令：

$export HCAT_HOME=$HIVE_HOME/hcatalog

$HCATJAR=$HCAT_HOME/share/hcatalog/hcatalog-core-0.11.0.jar

$ HCATPIGJAR=$HCAT_HOME/share/hcatalog/hive-hcatalog-pig-adapter-0.13.0.jar

美元出口 HADOOP_类路径=$HCATJAR:$HCATPIGJAR:$HIVE_HOME/lib/HIVE exec- 0.13.0.jar:$HIVE_HOME/lib/HIVE-metastore-0.13.0.jar:$HIVE_HOME/lib/jdo-api-3.0.1.jar:$HIVE_HOME/lib/libfb303-0.9.0.jar:$HIVE_HOME/lib/slf4j-api-1.6.4.jar:$HIVE_-HOME/conf:/usr/hadoop/hadoop-2.4.0/etc/hadoop/

$LIBJARS=

echo$HADOOP_CLASSPATH | sed-e's/：/，/g'

$export LIBJARS=$LIBJARS，$HIVE_HOME/lib/antlr-runtime-3.4.jar

我没有运行CDH发行版，但是通过以下配置设置，我能够完成此工作：

export HCAT_HOME=/usr/lib/hive-hcatalog
export HIVE_HOME=/usr/lib/hive
HCATJAR=$HCAT_HOME/share/hcatalog/hive-hcatalog-core-0.13.0.2.1.1.0-385.jar
HCATPIGJAR=$HCAT_HOME/share/hcatalog/hive-hcatalog-pig-adapter-0.13.0.2.1.1.0-385.jar
HIVE_VERSION=0.13.0.2.1.1.0-385
export HADOOP_CLASSPATH=$HCATJAR:$HCATPIGJAR:$HIVE_HOME/lib/hive-exec-$HIVE_VERSION.jar:$HIVE_HOME/lib/hive-metastore-$HIVE_VERSION.jar:$HIVE_HOME/lib/libfb303-0.9.0.jar:$HIVE_HOME/lib/libthrift-0.9.0.jar:$HIVE_HOME/conf:/etc/hadoop/conf
LIBJARS=`echo $HADOOP_CLASSPATH | sed -e 's/:/,/g'`
export LIBJARS=$LIBJARS,$HIVE_HOME/lib/antlr-runtime-3.4.jar

需要注意的几点：

最后一行“$LIBJARS”和“$HIVE_HOME”之间的逗号是正确的

我删除了对$HIVE_HOME/lib/jdo2-api-2.3-ec.jar和$HIVE_HOME/lib/slf4j-api-1.6.4.jar的引用，因为我的Hadoop发行版中没有这些引用。没有它，代码运行得很好

Hadoop的运行速度非常快，所以jar版本会发生变化。对于这些设置中引用的每个jar文件，执行

ls-l

命令以确保jar文件实际存在于您认为应该存在的位置

这段代码使用了一些不推荐使用的API调用。我的建议是（至少现在）不要更改代码。我发现，试图更改代码以使用未弃用的版本会破坏代码（另请参阅）

我希望这有帮助

确保类路径上有以下3个datanucleus JAR可用

datanucleus-rdbms-3.x.x.jar
datanucleus-core-3.x.x.jar
datanucleus-api-jdo-3.x.x.jar

另外，在HADOOP_类路径和类路径上有“$HIVE_HOME/conf”总是很好的，因为它有关于如何连接到元存储的重要信息。

您需要在~./bashrc中配置环境变量

export SQOOP_HOME=/usr/lib/sqoop
export HBASE_HOME=/usr/local/Hbase
export HIVE_HOME=/usr/local/hive
export HCAT_HOME=/usr/local/hive/hcatalog

您好，我已经尝试了上述解决方案，但仍然存在问题