Pulp

Problem¶

Say you have 4 workers and sync 10 things. 4 sync's start and 6 are still in the resource manager waiting for a worker to become free. Then the user submits a non-reserved task like orphan cleanup. This task immediately flows through the celery queue and is handed to one of the 4 workers who will work on it after handling the current sync. Then the resource manager assigns more work to free workers and one or more sync tasks could be stuck behind the orphan cleanup task.

Idea¶

Have all work flow through the resource_manager queue and have nothing flow through the celery queue. In fact have the workers stop consuming from the celery queue.

Design¶

Have tasks that current don't get a reservation, get a reservation that locks on a random uuid. This will cause the resource manager to not assign more work to a worker handling a task like an orphan cleanup.

Also get the workers to stop listening to the celery queue. There should be no ability to send work to the workers outside of the dedicated queues.