Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/image-processing/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark SqlContext导入和并行化Pypark时出错_Apache Spark_Dataframe_Pyspark_Rdd - Fatal编程技术网

Apache spark SqlContext导入和并行化Pypark时出错

Apache spark SqlContext导入和并行化Pypark时出错,apache-spark,dataframe,pyspark,rdd,Apache Spark,Dataframe,Pyspark,Rdd,我得到以下错误 TypeError:parallelize()缺少1个必需的位置参数:“c” 当从只有一列的字符串列表创建数据帧时,我还有一个问题: line = "Hello, world" sc.parallelize(list(line)).collect() 我得到以下错误: from pyspark.sql.types import * from pyspark.sql import SQLContext sqlContext = SQLContext(sc) schema = St

我得到以下错误

TypeError:parallelize()缺少1个必需的位置参数:“c”

当从只有一列的字符串列表创建数据帧时,我还有一个问题:

line = "Hello, world"
sc.parallelize(list(line)).collect()
我得到以下错误:

from pyspark.sql.types import *
from pyspark.sql import SQLContext
sqlContext = SQLContext(sc)
schema = StructType([StructField("name", StringType(), True)])
df3 = sqlContext.createDataFrame(fuzzymatchIntro, schema)
df3.printSchema()

提前感谢您

查看您的上述评论,您似乎以错误的方式初始化了
sparkContext


从pyspark.context导入SparkContext
从pyspark.sql.session导入SparkSession
sc=SparkContext
spark=SparkSession.builder.appName(“DFTest”).getOrCreate()

正确的方法是

----> 3 sqlContext = SQLContext(sc)
AttributeError: type object 'SparkContext' has no attribute '_jsc'

spark
对象可以完成
sqlContext

如何从pyspark.context从pyspark.sql.session导入SparkContext sc=SparkContext spark=SparkSession.builder.appName(“DFTest”).getOrCreate()创建
sc
from pyspark.sql.session import SparkSession
spark = SparkSession.builder.appName("DFTest").getOrCreate()
sc = spark.sparkContext