Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/scala/18.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Scala Spark submit Sql上下文创建语句不起作用_Scala_Apache Spark_Spark Streaming_Apache Spark Sql - Fatal编程技术网

Scala Spark submit Sql上下文创建语句不起作用

Scala Spark submit Sql上下文创建语句不起作用,scala,apache-spark,spark-streaming,apache-spark-sql,Scala,Apache Spark,Spark Streaming,Apache Spark Sql,我下面的代码不适用于Spark submit sqlContext.sql(s""" create external table if not exists landing ( date string, referrer string) partitioned by (partnerid string,dt string) row format delimited fields terminated by '\t' lines terminated by '\n' STORED AS TEXTF

我下面的代码不适用于Spark submit

sqlContext.sql(s"""
create external table if not exists landing (
date string,
referrer string)
partitioned by (partnerid string,dt string)
row format delimited fields terminated by '\t' lines terminated by '\n'
STORED AS TEXTFILE LOCATION 's3n://....'
      """)
它给出了错误: 线程“main”java.lang.RuntimeException中出现异常:[1.2]失败:`with''应为“”,但找到标识符create

此代码适用于Spark shell,但不适用于Spark submit。原因是什么?

默认情况下,spark shell中的“sqlContext”是“HiveContext”。也许您需要在脚本中新建一个HiveContext而不是sqlContext

你可以这样做:

import SparkContext._
import org.apache.spark.sql.hive._ 
val sc = new SparkContext()
val sqlContext = new HiveContext(sc)

可能缺少导入?也
导入sqlContext.implicits.\u
?类分析扩展可序列化的{val sc=new SparkContext()val sqlContext=SQLContextSingleton.getInstance(sc)导入sqlContext.implicits.\u=>我这样运行,但没有办法:(您最终找到了解决方案吗?