Python 在decorator类之外应用函数decorator';实例化范围?
我正在尝试修改在此处找到的代码库。该项目的目标是创建一个系统,该系统使用函数装饰器将函数添加到一个有向无环图中,该图表示要对某些给定输入执行的任务管道 我想通过创建一个Python 在decorator类之外应用函数decorator';实例化范围?,python,pipeline,python-decorators,Python,Pipeline,Python Decorators,我正在尝试修改在此处找到的代码库。该项目的目标是创建一个系统,该系统使用函数装饰器将函数添加到一个有向无环图中,该图表示要对某些给定输入执行的任务管道 我想通过创建一个Task类来修改项目,该类有一个方法流程,该方法经过修饰以执行管道中的任务,而不是像当前功能那样使用顶级功能 class Task: def __init__(self, name): self.name = name #@Pipeline.task() def process(self,
Task
类来修改项目,该类有一个方法流程
,该方法经过修饰以执行管道中的任务,而不是像当前功能那样使用顶级功能
class Task:
def __init__(self, name):
self.name = name
#@Pipeline.task()
def process(self, input):
return input
class AdditionTask(Task):
def __init__(self, name, value):
super().__init__(self, name)
self.value = value
@Pipeline.task()
def process(self, input):
return map(lambda x: x + self.value, input)
我假设这是一个有效的装饰,因为在描述Python装饰器时提供了example3.py
代码。尽管在尝试这样做时,在提供装饰器时似乎会产生一个错误
TypeError:task()缺少1个必需的位置参数:“self”
就我所知,AdditionTask
何时被实例化,函数decorator与Pipeline()
对象的实例化不在同一范围内。如以下SSCCE示例所示
from collections import deque
# https://github.com/vdymna/generic-python-pipeline/blob/master/pipeline/dag.py
class DAG:
"""Directed acyclic graph structure to manage pipeline task dependecies."""
def __init__(self):
self.graph = {}
def add(self, node, points_to=None):
"""Add new task not to the graph, specify optional 'points_to' parameter."""
if node not in self.graph:
self.graph[node] = []
if points_to:
if points_to not in self.graph:
self.graph[points_to] = []
self.graph[node].append(points_to) # todo: need to make sure not to add duplicates
# if sorted tasks and original graph lengths there must be a cycle
if len(self.sort()) != len(self.graph):
raise Exception('A cycle is detected in the graph')
def sort(self):
"""Sort all the task nodes based on the dependencies."""
self.in_degrees()
nodes_to_visit = deque()
for node, pointers in self.degrees.items():
# find all root nodes
if pointers == 0:
nodes_to_visit.append(node)
sorted_nodes = []
while nodes_to_visit:
node = nodes_to_visit.popleft()
for pointer in self.graph[node]:
self.degrees[pointer] -= 1
if self.degrees[pointer] == 0:
nodes_to_visit.append(pointer)
sorted_nodes.append(node)
return sorted_nodes
def in_degrees(self):
"""Determing number of in-coming edges for each task node."""
self.degrees = {}
for node in self.graph:
if node not in self.degrees:
self.degrees[node] = 0
for pointed in self.graph[node]:
if pointed not in self.degrees:
self.degrees[pointed] = 0
self.degrees[pointed] += 1
# https://github.com/vdymna/generic-python-pipeline/blob/master/pipeline/pipeline.py
class Pipeline:
"""Create a pipeline by chaining multiple tasks and identifying dependencies."""
def __init__(self):
self.tasks = DAG()
def task(self, depends_on=None):
"""Add new task to the pipeline and specify dependency task (optional)."""
def inner(func):
if depends_on:
self.tasks.add(depends_on, func)
else:
self.tasks.add(func)
return func
return inner
def run(self, *args):
"""Execute the pipeline and return each task results."""
sorted_tasks = self.tasks.sort()
completed = {}
for task in sorted_tasks:
for depend_on, nodes in self.tasks.graph.items():
if task in nodes:
completed[task] = task(completed[depend_on])
if task not in completed:
if sorted_tasks.index(task) == 0:
completed[task] = task(*args)
else:
completed[task] = task()
return completed
class Task:
def __init__(self, name):
self.name = name
#@Pipeline.task()
def process(self, input):
return input
class AdditionTask(Task):
def __init__(self, name, value):
super().__init__(self, name)
self.value = value
@Pipeline.task()
def process(self, input):
return map(lambda x: x + self.value, input)
if __name__ == "__main__":
pipeline = Pipeline()
add_op = AdditionTask(4)
print(pipeline.run(range(0, 4)))
是否有一种方法可以使用
管道中定义的函数装饰器来装饰类函数,从而将类函数包装到管道的功能中?
我现在唯一可能的解决方案是在
管道
中创建一个类global,也就是说,我将利用管道任务
,而不是定义自我任务
,这样,管道中添加的任务
独立于作用域。但是,如果我想创建多个管道
s,以及在管道任务和DAG的管理中,这会有问题
编辑:嗯,我想我想要一些类似于最后一个高级装饰器示例的东西。这里创建了一个应用于类的装饰器。装饰器检查装饰类的属性,然后将另一个装饰器应用于这些属性。在本例中,是一个决定函数运行时的装饰器。从岗位上,
def time_this(original_function):
print "decorating"
def new_function(*args,**kwargs):
print "starting timer"
import datetime
before = datetime.datetime.now()
x = original_function(*args,**kwargs)
after = datetime.datetime.now()
print "Elapsed Time = {0}".format(after-before)
return x
return new_function
def time_all_class_methods(Cls):
class NewCls(object):
def __init__(self,*args,**kwargs):
self.oInstance = Cls(*args,**kwargs)
def __getattribute__(self,s):
"""
this is called whenever any attribute of a NewCls object is accessed. This function first tries to
get the attribute off NewCls. If it fails then it tries to fetch the attribute from self.oInstance (an
instance of the decorated class). If it manages to fetch the attribute from self.oInstance, and
the attribute is an instance method then `time_this` is applied.
"""
try:
x = super(NewCls,self).__getattribute__(s)
except AttributeError:
pass
else:
return x
x = self.oInstance.__getattribute__(s)
if type(x) == type(self.__init__): # it is an instance method
return time_this(x) # this is equivalent of just decorating the method with time_this
else:
return x
return NewCls
#now lets make a dummy class to test it out on:
@time_all_class_methods
class Foo(object):
def a(self):
print "entering a"
import time
time.sleep(3)
print "exiting a"
oF = Foo()
oF.a()
尽管这仍然是我在原始发布文本中指出的相同问题,但我看不到将
管道
实例传递给time\u all\u class\u methods
所描述的装饰器的方法。虽然我可以使用类似于time\u all\u class\u methods
的类来修饰任务
类,但它仍然不知道将添加修饰属性的管道
的给定实例化实例;toPipeline.tasks
Pipeline.task
是一种未绑定的方法(即常规函数)。您需要一些管道的实例来调用它。