Pyspark Pypark can';t将浮点转换为浮点:-/

Pyspark Pypark can';t将浮点转换为浮点:-/,pyspark,Pyspark,我有一个pyspark rdd: proba_classe_0.take(2) [0.38030685472943737, 0.34728188900913715] 我想在DF上转换: from pyspark.sql.types import FloatType fields = [ StructField('probabilite' , FloatType() ) ] schema = StructType(fields) df_proba_classe_1 = spark.createD

我有一个pyspark rdd:

proba_classe_0.take(2)
[0.38030685472943737, 0.34728188900913715]
我想在DF上转换:

from pyspark.sql.types import FloatType
fields = [ StructField('probabilite' , FloatType() ) ]
schema = StructType(fields)
df_proba_classe_1 = spark.createDataFrame(proba_classe_1, schema=schema)
df_proba_classe_1.count()
我犯了一个奇怪的错误:

TypeError: StructType can not accept object 0.6196931452705625 in type <class 'float'>
TypeError:StructType无法接受类型中的对象0.6196931452705625

您必须映射rdd,因为rdd是字符串类型

rdd = sc\
.parallelize(['0.38030685472943737', '0.34728188900913715'])\
.map(lambda x: float(x))

df = spark\
.createDataFrame(rdd, FloatType()).toDF("id")

df.show()

+----------+
|        id|
+----------+
|0.38030684|
| 0.3472819|
+----------+

df.printSchema()

root
 |-- id: float (nullable = true)

谢谢,您知道如何将2个rdd中的2列并列到一个df中吗?