python和芹菜:覆盖用于gevent池的硬超时
有没有办法克服芹菜中的硬超时?我知道我可以通过失败作业的任务继承来实现这一点python和芹菜:覆盖用于gevent池的硬超时,python,class,error-handling,overriding,celery,Python,Class,Error Handling,Overriding,Celery,有没有办法克服芹菜中的硬超时?我知道我可以通过失败作业的任务继承来实现这一点 class MyTask(Task): def on_failure(self, exc, task_id, args, kwargs, einfo): print('{0!r} failed: {1!r}'.format(task_id, exc)) @app.task(base=MyTask, soft_time_limit=5, time_limit=10) def add(x, y): rai
class MyTask(Task):
def on_failure(self, exc, task_id, args, kwargs, einfo):
print('{0!r} failed: {1!r}'.format(task_id, exc))
@app.task(base=MyTask, soft_time_limit=5, time_limit=10)
def add(x, y):
raise KeyError()
但硬超时并不是失败的作业。我之所以想这样做,是因为软超时不适用于gevent池,而只适用于硬超时 我花了一点时间才弄明白,但你就是这样做的。从请求继承,然后从任务继承。从MyTask调用请求(on_failure方法所在的位置)
class MyRequest(Request):
def on_timeout(self, soft, timeout):
super(MyRequest, self).on_timeout(soft, timeout)
if not soft:
logger.warning(
'A hard timeout was enforced for task %s',
self.task.name
)
class MyTask(Task):
Request = MyRequest # you can use a FQN 'my.package:MyRequest'
def on_failure(self, exc, task_id, args, kwargs, einfo):
print('{0!r} failed: {1!r}'.format(task_id, exc))
def run_time_job():
a = random.randrange(0, 20)
print('sleeping for', a)
time.sleep(a)
@app.task(base=MyTask, soft_time_limit=5, time_limit=10)
def add(x, y):
results = None
try:
run_time_job()
results = x + y
except SoftTimeLimitExceeded:
print('time limit exceeded')
redis_db.sadd('failed_jobs', 'failed at {} + {}'.format(x, y))
except TimeLimitExceeded:
raise KeyError()
return results