Apache spark Apache spark 2.3在Apache HBase 2.0上的应用
需要在HBase上添加火花连接器,其中 Spark版本:2.3.1Apache spark Apache spark 2.3在Apache HBase 2.0上的应用,apache-spark,hbase,Apache Spark,Hbase,需要在HBase上添加火花连接器,其中 Spark版本:2.3.1 HBase版本:2.0.0 获取以下异常: Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.deploy.SparkHadoopUtil.getCurrentUserCredentials()Lorg/apache/hadoop/security/Credentials; at org.apache.hadoop.hbase
HBase版本:2.0.0 获取以下异常:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.deploy.SparkHadoopUtil.getCurrentUserCredentials()Lorg/apache/hadoop/security/Credentials;
at org.apache.hadoop.hbase.spark.HBaseContext.<init>(HBaseContext.scala:68)
at org.apache.hadoop.hbase.spark.JavaHBaseContext.<init>(JavaHBaseContext.scala:46)
at com.cloud.databaseroot.hbase.spark.JavaHBaseBulkPutExample.main(JavaHBaseBulkPutExample.java:60)
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-spark</artifactId>
<version>2.0.0-alpha4</version>
<exclusions>
<exclusion>
<artifactId>jackson-module-scala_2.10</artifactId>
<groupId>com.fasterxml.jackson.module</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.17.Final</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.11</artifactId>
<version>2.8.8</version>
</dependency>
线程“main”java.lang.NoSuchMethodError中的异常:org.apache.spark.deploy.SparkHadoopUtil.getCurrentUserCredentials()Lorg/apache/hadoop/security/Credentials;
位于org.apache.hadoop.hbase.spark.HBaseContext.(HBaseContext.scala:68)
位于org.apache.hadoop.hbase.spark.JavaHBaseContext.(JavaHBaseContext.scala:46)
位于com.cloud.databaseroot.hbase.spark.JavaHBaseBulkPutExample.main(JavaHBaseBulkPutExample.java:60)
从pom.xml捕捉:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.deploy.SparkHadoopUtil.getCurrentUserCredentials()Lorg/apache/hadoop/security/Credentials;
at org.apache.hadoop.hbase.spark.HBaseContext.<init>(HBaseContext.scala:68)
at org.apache.hadoop.hbase.spark.JavaHBaseContext.<init>(JavaHBaseContext.scala:46)
at com.cloud.databaseroot.hbase.spark.JavaHBaseBulkPutExample.main(JavaHBaseBulkPutExample.java:60)
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-spark</artifactId>
<version>2.0.0-alpha4</version>
<exclusions>
<exclusion>
<artifactId>jackson-module-scala_2.10</artifactId>
<groupId>com.fasterxml.jackson.module</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.17.Final</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.11</artifactId>
<version>2.8.8</version>
</dependency>
org.apache.hbase
hbase spark
2.0.0-4
jackson-module-scala_2.10
com.fasterxml.jackson.module
伊奥·内蒂
讨厌的
4.1.17.最终版本
com.fasterxml.jackson.module
jackson-module-scala_2.11
2.8.8
请告诉我哪里出错。hbase spark 2.0.0-alpha4版似乎与spark 2.3.1不兼容。 SparkhadoOptil.getCurrentUserCredentials方法在Spark版本中可用