Task #3076
Updated by dalley about 7 years ago
Pulp celerybeat can be removed from the Pulp3 architecture entirely if we can transition its two remaining software responsibilities to the workers themselves. Specifically: * Celerybeat looks to find missing workers and call "mark_worker_offline()". That call effectively cancels any reserved tasks assigned to the worker and removes the worker's records from the database. * Check that at least one resource_manager process is running and that at least one worker process is running and log loudly as necessary if they aren't. These responsibilities should move to the worker heartbeat code "here":https://github.com/bmbouter/pulp/blob/0fdd2515bbd49094bf3e0782fa3a6df430f3e9a2/server/pulp/server/async/app.py#L109-L123 All workers should effectively run this cleanup whenever they see it needs to be done, this makes a shared responsibility to across all workers. The "scheduler.py":https://github.com/bmbouter/pulp/blob/63aa4d9ce5de374d01d8bfdcda3f84594de4e5af/server/pulp/server/async/scheduler.py should be deleted along with any orphaned code that was exclusively used by scheduler.py. This will require a few other updates too: * searching and updating updateing the documentation to remove celerybeat references * updating the devel environment to not deal with celerybeat or its units * updating the galaxy playbooks to not deal with celerybeat or its units. Also there are two correctness points to verify to ensure that any old records won't cause correctness problems: * Verify the status API filters out records older than 30 seconds * Verify the resource manager filters out workers who haven't checked in within 30 seconds
.