使用Java JDBC访问配置单元问题-无法从ZooKeeper读取HiveServer2 uri_Hive

使用Java JDBC访问配置单元问题-无法从ZooKeeper读取HiveServer2 uri

hive

使用Java JDBC访问配置单元问题-无法从ZooKeeper读取HiveServer2 uri,hive,Hive,使用Beeline，我能够成功连接 !connect jdbc:hive2://xxxxxxx/;serviceDiscoveryMode=zookeeper;zookeeperNameSpace=hiveserver2 当我开始使用具有相同url的Java Jdbc代码时，出现以下错误错误日志： Exception in thread "main" java.sql.SQLException: org.apache.hive.jdbc.ZooKeeperHiveClientExcepti

使用Beeline，我能够成功连接

!connect jdbc:hive2://xxxxxxx/;serviceDiscoveryMode=zookeeper;zookeeperNameSpace=hiveserver2

当我开始使用具有相同url的Java Jdbc代码时，出现以下错误

错误日志：

 Exception in thread "main" java.sql.SQLException: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 uri from ZooKeeper
                    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:131)
                    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
                    at java.sql.DriverManager.getConnection(DriverManager.java:664)
                    at java.sql.DriverManager.getConnection(DriverManager.java:270)
                    at io.saagie.example.hive.HiveJdbc.main(HiveJdbc.java:23)
    Caused by: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 uri from ZooKeeper
                    at org.apache.hive.jdbc.ZooKeeperHiveClientHelper.getNextServerUriFromZooKeeper(ZooKeeperHiveClientHelper.java:86)
                    at org.apache.hive.jdbc.Utils.resolveAuthorityUsingZooKeeper(Utils.java:517)
                    at org.apache.hive.jdbc.Utils.resolveAuthority(Utils.java:489)
                    at org.apache.hive.jdbc.Utils.parseURL(Utils.java:396)
                    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:129)
                    ... 4 more
    Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hiveserver2
                    at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
                    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
                    at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1590)
                    at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:214)
                    at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:203)
                    at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
                    at org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:199)
                    at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:191)
                    at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:38)
                    at org.apache.hive.jdbc.ZooKeeperHiveClientHelper.getNextServerUriFromZooKeeper(ZooKeeperHiveClientHelper.java:69)
                    ... 8 more

POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>io.vk</groupId>
    <artifactId>java-read-and-write-from-hive</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <hadoop.version>2.6.0</hadoop.version>
        <log4j.version>1.2.17</log4j.version>
        <hive-jdbc.version>1.2.0</hive-jdbc.version>
    </properties>

    <dependencies>
        <!-- Hadoop main client artifact -->
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>${hadoop.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-jdbc</artifactId>
            <version>${hive-jdbc.version}</version>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>${log4j.version}</version>
        </dependency>
      <!--
        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-service</artifactId>
            <version>1.2.1000.2.4.2.10-1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-jdbc</artifactId>
            <version>1.2.1000.2.4.2.10-1</version>
        </dependency>

      -->
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>io.saagie</groupId>
                <artifactId>saagie-maven-plugin</artifactId>
                <version>1.0.2</version>
                <configuration>
                    <platformId>1</platformId>
                    <jobName>example-java-read-and-write-from-hive</jobName>
                    <jobCategory>extract</jobCategory>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <archive>
                        <manifest>
                            <mainClass>io.vk.example.hive.Main</mainClass>
                        </manifest>
                    </archive>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id> <!-- this is used for inheritance merges -->
                        <phase>package</phase> <!-- bind to the packaging phase -->
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>


4.0.0
io.vk
java从配置单元读写
1.0-快照
UTF-8
2.6.0
1.2.17
1.2.0
org.apache.hadoop
hadoop客户端
${hadoop.version}
org.apache.hive
蜂窝jdbc
${hive jdbc.version}
log4j
log4j
${log4j.version}
伊奥·萨吉
saagie maven插件
1.0.2
1.
示例java从配置单元读写
摘录
org.apache.maven.plugins
maven编译器插件
1.8
1.8
maven汇编插件
io.vk.example.hive.Main
带有依赖项的jar
组装
包裹
单一的

我已经解决了这个问题，共享ans以帮助他人

我们可以把我的问题分成两部分

1）我在JavaJDBC中遇到以下异常，当我在直线上使用相同的url时，试图连接窗口机器

Exception in thread "main" java.sql.SQLException: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 uri from ZooKeeper
                    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:131)
                    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
                    at java.sql.DriverManager.getConnection(DriverManager.java:664)
                    at java.sql.DriverManager.getConnection(DriverManager.java:270)
                    at io.saagie.example.hive.HiveJdbc.main(HiveJdbc.java:23)
    Caused by: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 uri from ZooKeeper
                    at org.apache.hive.jdbc.ZooKeeperHiveClientHelper.getNextServerUriFromZooKeeper(ZooKeeperHiveClientHelper.java:86)
                    at org.apache.hive.jdbc.Utils.resolveAuthorityUsingZooKeeper(Utils.java:517)
                    at org.apache.hive.jdbc.Utils.resolveAuthority(Utils.java:489)
                    at org.apache.hive.jdbc.Utils.parseURL(Utils.java:396)
                    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:129)
                    ... 4 more
    Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hiveserver2
                    at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
                    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
                    at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1590)
                    at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:214)
                    at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:203)
                    at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
                    at org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:199)
                    at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:191)
                    at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:38)
                    at org.apache.hive.jdbc.ZooKeeperHiveClientHelper.getNextServerUriFromZooKeeper(ZooKeeperHiveClientHelper.java:69)
                    ... 8 more

在我的情况下，端口被防火墙阻止，这就是我得到上述异常的原因

2）现在，一旦端口打开，我再次尝试运行我的程序，并开始得到不同的错误

 Exception in thread "main" java.lang.NullPointerException at 
   org.apache.thrift.transport.TSocket.open(TSocket.java:170) at 
   org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266) at 

 org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at 
  org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:204) at 
  org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:176) at 
  org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at 
  java.sql.DriverManager.getConnection(DriverManager.java:571) at 
  java.sql.DriverManager.getConnection(DriverManager.java:215) at 
  com.hugedata.hive.HiveJdbcConnect.getConnection(HiveJdbcConnect.java:19) at 
  com.hugedata.hive.HiveTest.main(HiveTest.java:12)

线程“main”java.lang.NullPointerException中的异常位于 org.apache.thrift.transport.TSocket.open（TSocket.java:170）位于 org.apache.thrift.transport.TSaslTransport.open（TSaslTransport.java:266）位于 org.apache.thrift.transport.TSaslClientTransport.open（TSaslClientTransport.java:37）位于 org.apache.hive.jdbc.HiveConnection.openTransport（HiveConnection.java:204）位于 HiveConnection.（HiveConnection.java:176）位于 org.apache.hive.jdbc.HiveDriver.connect（HiveDriver.java:105）位于 java.sql.DriverManager.getConnection（DriverManager.java:571）位于 java.sql.DriverManager.getConnection（DriverManager.java:215）位于 com.hugedata.hive.HiveJdbcConnect.getConnection（HiveJdbcConnect.java:19）位于 com.hugedata.hive.HiveTest.main（HiveTest.java:12）经过分析发现，需要将驱动程序jar版本与配置单元版本匹配

我使用的是1.2.0版本

     <dependency>
        <groupId>org.apache.hive</groupId>
        <artifactId>hive-jdbc</artifactId>
        <version>1.2.0</version>
   </dependency>


org.apache.hive
蜂窝jdbc
1.2.0

更新为2.0.0，并开始工作

 <dependency>
        <groupId>org.apache.hive</groupId>
        <artifactId>hive-jdbc</artifactId>
        <version>2.0.0</version>
    </dependency>


org.apache.hive
蜂窝jdbc
2.0.0