Project

Profile

Help

Story #21

As a user, Pulp won't get into trouble if I accidentally run celerybeat for a long time with no resource manager

Added by Anonymous almost 6 years ago. Updated over 1 year ago.

Status:
CLOSED - WORKSFORME
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

There are two things we can do to help this scenario:1) Have the babysit() function remember when it last babysat. If it hasn't been at least a minute since then, return. This will have the effect of letting us burn through the queue very quickly when there are a pile of tasks in it.2) Have the babysit() function accept the massaged output of the Celery active_queues() call as input. This Celery function takes on the order of seconds to complete, and could be done by any of the workers in the system. Whatever worker performs the work could also reduce the data structure down to just the workers and the queue names they are subscribed to so that the message can remain small. This will help the resource manager be more responsive in general as an added benefit.

History

#2 Updated by bmbouter almost 4 years ago

  • Status changed from NEW to CLOSED - WORKSFORME

A change [0] that was made a while ago made it so that a user who accidentally runs Pulp without a resource manager should not get in trouble. Tasks will be accepted, status of those tasks will be accurate, and once the resource manager starts tasks will process normally.

[0]: https://github.com/pulp/pulp/commit/eb295a9a14a0369a38c3401a99840a749a4d613a

#3 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF