As a user, Pulp won't get into trouble if I accidentally run celerybeat for a long time with no resource manager
There are two things we can do to help this scenario:1) Have the babysit() function remember when it last babysat. If it hasn't been at least a minute since then, return. This will have the effect of letting us burn through the queue very quickly when there are a pile of tasks in it.2) Have the babysit() function accept the massaged output of the Celery active_queues() call as input. This Celery function takes on the order of seconds to complete, and could be done by any of the workers in the system. Whatever worker performs the work could also reduce the data structure down to just the workers and the queue names they are subscribed to so that the message can remain small. This will help the resource manager be more responsive in general as an added benefit.
#2 Updated by bmbouter almost 5 years ago
- Status changed from NEW to CLOSED - WORKSFORME
A change  that was made a while ago made it so that a user who accidentally runs Pulp without a resource manager should not get in trouble. Tasks will be accepted, status of those tasks will be accurate, and once the resource manager starts tasks will process normally.
Please register to edit this issue