Project

Profile

Help

Story #21

closed

As a user, Pulp won't get into trouble if I accidentally run celerybeat for a long time with no resource manager

Added by Anonymous about 9 years ago. Updated almost 5 years ago.

Status:
CLOSED - WORKSFORME
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

There are two things we can do to help this scenario:1) Have the babysit() function remember when it last babysat. If it hasn't been at least a minute since then, return. This will have the effect of letting us burn through the queue very quickly when there are a pile of tasks in it.2) Have the babysit() function accept the massaged output of the Celery active_queues() call as input. This Celery function takes on the order of seconds to complete, and could be done by any of the workers in the system. Whatever worker performs the work could also reduce the data structure down to just the workers and the queue names they are subscribed to so that the message can remain small. This will help the resource manager be more responsive in general as an added benefit.

Actions #1

Updated by rbarlow about 9 years ago

  • Project changed from 22 to Pulp
Actions #2

Updated by bmbouter about 7 years ago

  • Status changed from NEW to CLOSED - WORKSFORME

A change [0] that was made a while ago made it so that a user who accidentally runs Pulp without a resource manager should not get in trouble. Tasks will be accepted, status of those tasks will be accurate, and once the resource manager starts tasks will process normally.

[0]: https://github.com/pulp/pulp/commit/eb295a9a14a0369a38c3401a99840a749a4d613a

Actions #3

Updated by bmbouter almost 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF