Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/351.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 无法将StructField与PySpark一起使用_Python_Apache Spark_Pyspark - Fatal编程技术网

Python 无法将StructField与PySpark一起使用

Python 无法将StructField与PySpark一起使用,python,apache-spark,pyspark,Python,Apache Spark,Pyspark,我正在运行PySpark shell,无法创建数据帧。我做完了 import pyspark from pyspark.sql.types import StructField from pyspark.sql.types import StructType 返回的所有数据都没有任何错误 然后我尝试运行以下命令: schemaString = "name age" fields = [StructField(field_name, StringType(), True) for field_n

我正在运行PySpark shell,无法创建数据帧。我做完了

import pyspark
from pyspark.sql.types import StructField
from pyspark.sql.types import StructType
返回的所有数据都没有任何错误

然后我尝试运行以下命令:

schemaString = "name age"
fields = [StructField(field_name, StringType(), True) for field_name in schemaString.split()]
并不断获取错误:'name'StructField'未定义

基本上,我在这里遵循Spark文档:

奇怪的是,如果我删除
for
循环并执行此操作,它会工作:

fields = [StructField('field1', StringType(), True)]

它适用于以下代码。和的文件。而1.3已经很老了

from pyspark.sql.types import *
schemaString = "name age"

fields = [StructField(field_name, StringType(), True) 
    for field_name in schemaString.split()]

也许您应该导入如下所示的包:

    from pyspark import SparkContext, SparkConf
    from pyspark.sql import SparkSession
    from pyspark.sql import Row
    from pyspark.sql.types import StructField
    from pyspark.sql.types import StructType
    from pyspark.sql.types import StringType

我会再次检查您是否确实正确导入了
import
ed
StructField
。啊,是的,看起来我必须从pyspark.sql.types import*中导入
,而不是从pyspark.sql import*