Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python PySpark对self-inside map()和reduce()函数的引用_Python_Apache Spark_Mapreduce_Pyspark - Fatal编程技术网

Python PySpark对self-inside map()和reduce()函数的引用

Python PySpark对self-inside map()和reduce()函数的引用,python,apache-spark,mapreduce,pyspark,Python,Apache Spark,Mapreduce,Pyspark,如前所述,我应该避免在map函数中调用self。在此基础上,我有两个问题: 让我们使用此处所述的相同代码: class C0(object): def func0(self, arg): # added self ... def func1(self, rdd): # added self func = self.func0 result = rdd.map(lambda x: func(x)) result=rdd.map(lambda x:func(x)

如前所述,我应该避免在map函数中调用self。在此基础上,我有两个问题: 让我们使用此处所述的相同代码:

class C0(object):

  def func0(self, arg): # added self
    ...

  def func1(self, rdd): # added self
    func = self.func0
    result = rdd.map(lambda x: func(x))
  • result=rdd.map(lambda x:func(x))

    result=rdd.map(func)
    ?特别是在我以前使用
    func=self.func0
    的情况下

  • 假设func0从类中调用另一个方法:

  • C0类(对象):


    Spark是如何处理这个问题的?我应该在func0中执行
    func2=self.func2
    吗?

    我想这就是我想要的。非常感谢你
      def func2(self, arg):
        ...
    
      def func0(self, arg): # added self
        self.func2(arg)
        ...
    
      def func1(self, rdd): # added self
        func = self.func0
        result = rdd.map(lambda x: func(x))