Project

Profile

Help

Story #21

closed

As a user, Pulp won't get into trouble if I accidentally run celerybeat for a long time with no resource manager

Added by Anonymous over 9 years ago. Updated almost 5 years ago.

Status:
CLOSED - WORKSFORME
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

There are two things we can do to help this scenario:1) Have the babysit() function remember when it last babysat. If it hasn't been at least a minute since then, return. This will have the effect of letting us burn through the queue very quickly when there are a pile of tasks in it.2) Have the babysit() function accept the massaged output of the Celery active_queues() call as input. This Celery function takes on the order of seconds to complete, and could be done by any of the workers in the system. Whatever worker performs the work could also reduce the data structure down to just the workers and the queue names they are subscribed to so that the message can remain small. This will help the resource manager be more responsive in general as an added benefit.

Also available in: Atom PDF