Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/delphi/8.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark 使用IntelliJ IDEA在本地模式下运行Spark连接蜂箱_Apache Spark_Intellij Idea_Hive_Localhost - Fatal编程技术网

Apache spark 使用IntelliJ IDEA在本地模式下运行Spark连接蜂箱

Apache spark 使用IntelliJ IDEA在本地模式下运行Spark连接蜂箱,apache-spark,intellij-idea,hive,localhost,Apache Spark,Intellij Idea,Hive,Localhost,我试图在IntelliJ上使用Scala/Spark代码来读取在Hive上创建的表。我已经在Ubuntu上安装了Hadoop和Hive,它们都在我的本地环境中正确运行 下面是我试图在IntelliJ上运行的代码: import org.apache.spark.sql.SparkSession // warehouseLocation points to the default location for managed databases and tables val warehouseLoc

我试图在IntelliJ上使用Scala/Spark代码来读取在Hive上创建的表。我已经在Ubuntu上安装了Hadoop和Hive,它们都在我的本地环境中正确运行

下面是我试图在IntelliJ上运行的代码:

import org.apache.spark.sql.SparkSession

// warehouseLocation points to the default location for managed databases and tables
val warehouseLocation = "hdfs:///user/hive/warehouse"

val spark = SparkSession
 .builder()
 .appName("Spark Hive Local Connector")
 .config("spark.sql.warehouse.dir", warehouseLocation)
 .config("spark.master", "local[*]")
 .enableHiveSupport()
 .getOrCreate()

import spark.implicits._
import spark.sql

sql("SELECT * FROM test.employee").show()
执行此代码后,我有一个错误,表示找不到该表。我还尝试将“warehouseLocation”变量设置为“/user/hive/warehouse”,但它仍然不起作用

我想知道是否有必要在hive-site.xml文件中配置metastore,然后将hive.metastore.uri添加到SparkSession的配置参数中?欢迎提出其他建议

谢谢你的帮助