Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/328.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
制作一个Python装饰器,将任何@classmethod函数转换为并行处理值向量的函数_Python_Python 3.x_Multiprocessing_Decorator_Vectorization - Fatal编程技术网

制作一个Python装饰器,将任何@classmethod函数转换为并行处理值向量的函数

制作一个Python装饰器,将任何@classmethod函数转换为并行处理值向量的函数,python,python-3.x,multiprocessing,decorator,vectorization,Python,Python 3.x,Multiprocessing,Decorator,Vectorization,这是一个基本的Python问题。我正试图用我自己的装饰器来装饰一个@classmethod,它将任何函数转换为一个函数,该函数接受原始函数的参数列表作为参数。这里有一个例子 import multiprocessing as mp def my_decorator(method): def method_in_vector_form(obj, L, *args, **kwargs): pool = mp.Pool(2) print('Starting d

这是一个基本的Python问题。我正试图用我自己的装饰器来装饰一个
@classmethod
,它将任何函数转换为一个函数,该函数接受原始函数的参数列表作为参数。这里有一个例子

import multiprocessing as mp

def my_decorator(method):
    def method_in_vector_form(obj, L, *args, **kwargs):
        pool = mp.Pool(2)
        print('Starting decorator')
        results = [pool.apply_async(method, args=(obj, value)) for value in L]
        pool.close()
        pool.join()
        return [r.get() for r in results]
    return method_in_vector_form

class Foo(object):        
    @classmethod
    def boo(cls, i):
        print('Running from Foo\'s boo' + i)

    @classmethod
    @my_decorator
    def bar(cls, chunk_id):
        print('Running from Foo\'s bar')
        cls.boo(chunk_id)
        return chunk_id * 2

class Child(Foo):
    @classmethod
    def boo(cls, i):
        print('Running from Child\'s version of boo' + i)

if __name__ == '__main__':
    print(Foo.bar([1, 2, 3]))
    print(Child.bar([1, 2, 3]))
您有一个函数
foo()
,如果您想在参数上按顺序运行它,
a
b
c
。你会的

result_a = foo(a)
result_b = foo(b)
result_c = foo(c)
您只需创建一个在参数列表上工作的函数,
L=[a,b,c]
,然后您就可以

results = foo_on_list([a, b, c])
我想做一个decorator,对任何函数
foo()
都能做到这一点。更重要的是,我希望使用
多处理
库能够异步执行
foo(a)
foo(b)
。我写了一个很好的修饰符,它适用于静态函数,但是我很难推广到必须使用
cls
self
参数的函数。这里有一个例子

import multiprocessing as mp

def my_decorator(method):
    def method_in_vector_form(obj, L, *args, **kwargs):
        pool = mp.Pool(2)
        print('Starting decorator')
        results = [pool.apply_async(method, args=(obj, value)) for value in L]
        pool.close()
        pool.join()
        return [r.get() for r in results]
    return method_in_vector_form

class Foo(object):        
    @classmethod
    def boo(cls, i):
        print('Running from Foo\'s boo' + i)

    @classmethod
    @my_decorator
    def bar(cls, chunk_id):
        print('Running from Foo\'s bar')
        cls.boo(chunk_id)
        return chunk_id * 2

class Child(Foo):
    @classmethod
    def boo(cls, i):
        print('Running from Child\'s version of boo' + i)

if __name__ == '__main__':
    print(Foo.bar([1, 2, 3]))
    print(Child.bar([1, 2, 3]))
我希望得到以下输出(可能由于异步性而无序)

这是电流输出

Starting decorator
Traceback (most recent call last):
  File "C:\....\dec_example.py", line 36, in <module>
    print(Foo.bar([1, 2, 3]))
  File "C:\....\TextFromPdf\dec_example.py", line 10, in method_in_vector_form
    return [r.get() for r in results]
  File "C:\....\dec_example.py", line 10, in <listcomp>
    return [r.get() for r in results]
  File "C:\....\pool.py", line 599, in get
    raise self._value
  File "C:\....\pool.py", line 383, in _handle_tasks
    put(task)
  File "C:\....\connection.py", line 206, in send
    self._send_bytes(ForkingPickler.dumps(obj))
  File "C:\....\reduction.py", line 50, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function Foo.bar at 0x0000000002897158>: attribute lookup bar on __main__ failed
启动装饰程序
回溯(最近一次呼叫最后一次):
文件“C:\..\dec\u example.py”,第36行,在
打印(Foo.bar([1,2,3]))
文件“C:\..\TextFromPdf\dec\u example.py”,第10行,方法为矢量形式
返回[r.get(),用于结果中的r]
文件“C:\..\dec\u example.py”,第10行,在
返回[r.get(),用于结果中的r]
get中第599行的文件“C:\..\pool.py”
提升自我价值
文件“C:\..\pool.py”,第383行,在任务处理中
放置(任务)
文件“C:\..\connection.py”,第206行,在send中
self.\u发送字节(ForkingPickler.dumps(obj))
文件“C:\..\reduce.py”,第50行,转储
cls(buf,协议).dump(obj)
_pickle.PicklingError:无法pickle:main上的属性查找栏失败

该问题/答案与编写自定义装饰程序无关。您确定希望从Foo's bar运行的
只出现一次吗?它不会出现在每次调用
boo
之前吗?这里的问题不在于装饰程序,而在于进程间通信所需的序列化(酸洗)。