Refactor #1084
Updated by bmbouter almost 9 years ago
Historically, Pulp has used the celery Results backend, but I don't think its needed anymore. By using it we are having mongoDB issues (see #956). The results backend was used in Pulp 2.4 so the resource_manager could tell httpd which queue to send a task into but that is being done differently now. h2. Deliverables * Remove the "create_mongo_config()":https://github.com/pulp/pulp/blob/01fcf261c38f9b4b057839980f892f85a8697a27/server/pulp/server/async/celery_instance.py#L36-L70 function and it's associated tests * Have Pulp stop having celery connect to MongoDB at all by removing "the mongodb and results config":https://github.com/pulp/pulp/blob/01fcf261c38f9b4b057839980f892f85a8697a27/server/pulp/server/async/celery_instance.py#L75-L76 and fixup associated tests. * Rework the FailureWatcher feature as described below * Ensure that the "TaskResult object":https://github.com/pulp/pulp/blob/dd5e47b44f0ae126d41ba2a49a9882425cbd9fbd/server/pulp/server/async/tasks.py#L202 works as expected without the results backend present. * Write a migration to delete the celery_taskmeta collection * Write a release note for this change * Run the integration suite before and after and ensure that this change does not break anything. h2. FailureWatcher The results backend is used "here in the FailureWatcher thread of celerybeat":https://github.com/pulp/pulp/blob/04e33e4d6e856d50e0c47c96e0b15a17f1c05659/server/pulp/server/async/scheduler.py#L99. The call to AsyncResult().result will fail when the results backend is removed so this needs to be done differently. I suggest the following: * Delete "the FailureWatcher":https://github.com/pulp/pulp/blob/04e33e4d6e856d50e0c47c96e0b15a17f1c05659/server/pulp/server/async/scheduler.py#L35-L128 entirely. * Remove any vestige of the FailureWatcher from the rest of "scheduler.py":https://github.com/pulp/pulp/blob/04e33e4d6e856d50e0c47c96e0b15a17f1c05659/server/pulp/server/async/scheduler.py * Reimplement the auto-disabling of scheduled actions which fail N times by creating the appropriate logic in the "on_failure handler":https://github.com/pulp/pulp/blob/dd5e47b44f0ae126d41ba2a49a9882425cbd9fbd/server/pulp/server/async/tasks.py#L439 which is executed by Celery synchronously upon completion. * update/write tests accordingly. h2. QE instructions instructions: * Verify that the migration removes the celery_taskmeta collection * Verify the release notes * Verify that the fix which includes this change passes a full regression test
.