Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/295.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 如何pickle从PySpark DataFrame继承的对象实例_Python_Apache Spark_Pyspark_Pickle - Fatal编程技术网

Python 如何pickle从PySpark DataFrame继承的对象实例

Python 如何pickle从PySpark DataFrame继承的对象实例,python,apache-spark,pyspark,pickle,Python,Apache Spark,Pyspark,Pickle,因此,我有一个自定义类继承自pyspark.sql.dataframe.dataframe: from pyspark.sql.dataframe import DataFrame class MyCustomClass(DataFrame): def __init__(self, sdf): super().__init__(sdf._jdf, sdf.sql_ctx) 我希望能够定制此类实例的酸洗。我试图做的是重写\uuuu getstate\uuuuuuuuu

因此,我有一个自定义类继承自
pyspark.sql.dataframe.dataframe

from pyspark.sql.dataframe import DataFrame

class MyCustomClass(DataFrame):

    def __init__(self, sdf):
        super().__init__(sdf._jdf, sdf.sql_ctx)
我希望能够定制此类实例的酸洗。我试图做的是重写
\uuuu getstate\uuuuuuuuuuu()
\uuuuuuu setstate\uuuuuuuuuu()
方法:

def __getstate__(self):

    state = super().__getstate__()
    state['extra_key'] = self.one_property

    return state
但我得到了以下错误:

>       state = super().__getstate__()
E       AttributeError: 'super' object has no attribute '__getstate__'
这让我觉得
\uuu getstate\uuu
不是由父类(即PySpark数据帧)实现的。 你有什么想法吗