Project

Profile

Help

Refactor #1084

closed

Task #1014: Short Term Improvements for Pulp's use of MongoDB

Stop Pulp from using the Celery results backend

Added by bmbouter over 7 years ago. Updated over 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.7.0
Groomed:
No
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Quarter:

Description

This is the fix for issue #956. If you assign this to yourself, you also need to assign #956 because you'll be fixing that issue with these changes.

Historically, Pulp has used the celery Results backend, but I don't think its needed anymore. By using it we are having mongoDB issues (see #956). The results backend was used in Pulp 2.4 so the resource_manager could tell httpd which queue to send a task into but that is being done differently now.

Deliverables

  • Remove the create_mongo_config() function and it's associated tests
  • Have Pulp stop having celery connect to MongoDB at all by removing the mongodb and results config and fixup associated tests.
  • Rework the FailureWatcher feature as described below
  • Ensure that the TaskResult object works as expected without the results backend present.
  • Write a migration to delete the celery_taskmeta collection
  • Write a release note for this change
  • Run the integration suite before and after and ensure that this change does not break anything.

FailureWatcher

The results backend is used here in the FailureWatcher thread of celerybeat. The call to AsyncResult().result will fail when the results backend is removed so this needs to be done differently. I suggest the following:

  • Delete the FailureWatcher entirely.
  • Remove any vestige of the FailureWatcher from the rest of scheduler.py
  • Reimplement the auto-disabling of scheduled actions which fail N times by creating the appropriate logic in the on_failure handler which is executed by Celery synchronously upon completion.
  • update/write tests accordingly.

Related issues

Related to Pulp - Issue #956: Pulp's Celery result backend connection cannot use Mongo replica sets with automatic failoverCLOSED - CURRENTRELEASEdkliban@redhat.comActions
Actions #1

Updated by bmbouter over 7 years ago

  • Description updated (diff)
Actions #2

Updated by bmbouter over 7 years ago

  • Related to Issue #956: Pulp's Celery result backend connection cannot use Mongo replica sets with automatic failover added
Actions #3

Updated by bmbouter about 7 years ago

  • Description updated (diff)
Actions #4

Updated by mhrivnak about 7 years ago

  • Platform Release set to 2.6.4
  • Groomed changed from No to Yes
Actions #5

Updated by mhrivnak about 7 years ago

  • Tracker changed from Story to Refactor
Actions #6

Updated by bmbouter about 7 years ago

  • Description updated (diff)
Actions #7

Updated by mhrivnak about 7 years ago

  • Priority changed from Normal to High
Actions #8

Updated by bmbouter about 7 years ago

  • Platform Release deleted (2.6.4)
Actions #9

Updated by dkliban@redhat.com about 7 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dkliban@redhat.com
Actions #10

Updated by dkliban@redhat.com about 7 years ago

  • Status changed from ASSIGNED to POST

Added by dkliban@redhat.com about 7 years ago

Revision aa4d57df

Removes MongoDB as celery result backend

This patch also removes the FailureHandler which relied on checking the results backend to determine if a scheduled task needs to have it's schedule disabled after reaching a failure threshold. The logic is moved to the on_success and on_failure methods for Task defined in Pulp.

https://pulp.plan.io/issues/956 fixes #956 https://pulp.plan.io/issues/1084 fixes #1084

Added by dkliban@redhat.com about 7 years ago

Revision aa4d57df

Removes MongoDB as celery result backend

This patch also removes the FailureHandler which relied on checking the results backend to determine if a scheduled task needs to have it's schedule disabled after reaching a failure threshold. The logic is moved to the on_success and on_failure methods for Task defined in Pulp.

https://pulp.plan.io/issues/956 fixes #956 https://pulp.plan.io/issues/1084 fixes #1084

Actions #11

Updated by dkliban@redhat.com about 7 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #12

Updated by dkliban@redhat.com about 7 years ago

  • Platform Release set to 2.7.0
  • Groomed changed from Yes to No
Actions #13

Updated by dkliban@redhat.com almost 7 years ago

  • Status changed from MODIFIED to 5
Actions #14

Updated by rbarlow over 6 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #15

Updated by bmbouter over 3 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF