Apache spark Derby Metastore目录是在spark工作区中创建的

我已经安装了spark 2.1.0并与eclipse和hive2集成，并且在Mysql中配置了metastore，还将hive-site.xml文件放在spark>>conf文件夹中。我正在尝试从eclipse访问配置单元中已经存在的表。 当我执行程序metastore文件夹时，会在spark workspace和eclipse控制台中创建derby.log文件，并显示以下信息：

Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
17/06/13 18:26:43 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/06/13 18:26:43 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/06/13 18:26:43 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
17/06/13 18:26:43 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
17/06/13 18:26:43 INFO Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing
17/06/13 18:26:43 INFO MetaStoreDirectSql: Using direct SQL, underlying DB is MYSQL

spark无法定位已配置的mysql metastore数据库

也抛出了错误

Exception in thread "main" java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveSessionState':

代码：

Build.sbt

libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.1.0"
libraryDependencies += "com.typesafe" % "config" % "1.3.0" 
libraryDependencies += "org.apache.spark" % "spark-sql_2.11" % "2.1.0"
libraryDependencies += "org.apache.spark" % "spark-hive_2.11" % "2.1.0"
libraryDependencies += "mysql" % "mysql-connector-java" % "5.1.42"

注意：我可以从Spark shell访问配置单元表

谢谢

当您放置context.setMasterlocal时，它可能不会查找您在集群中设置的spark配置；特别是当您从ECLIPSE触发它时

用它做一个罐子；并作为spark submit-class-master从cmd触发spark://207.184.161.138:7077 -部署模式客户端

主ip：spark://207.184.161.138:7077 应替换为群集的ip和spark端口

并且，记住初始化HiveContext以触发对底层配置单元的查询

val hc = new HiveContext(sc)
hc.sql("SELECT * FROM ...")

---

如何从ECLIPSE触发spark作业？setMaster。。。不应该是“本地的”；并且应该指向application-master。用它制作一个罐子；并使用spark submit调用它。。。。这样，您将得到预期的结果。@raktopalbordoloi我在localhost中运行它，我给出了local[*]@raktopalbordoloi，我刚刚将hive-site.xml文件添加到eclipse的资源文件夹中，现在它指向MYSQL，但有一些错误，请查看更新后的问题。您能否共享spark core和spark hive的依赖关系矩阵@VickysterThank非常感谢您，我通过在eclipse资源文件夹中添加core-site.xml和hdfs-site.xml文件以及hive-site.xml文件获得了它。！

val hc = new HiveContext(sc)
hc.sql("SELECT * FROM ...")