Project

Profile

Help

Story #1

closed

Task #1014: Short Term Improvements for Pulp's use of MongoDB

As a user, I can have Pulp attempt use auto_retry application wide using the 'unsafe_autoretry' parameter

Added by Anonymous about 9 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
2.7.0
Groomed:
No
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Quarter:

Description

Pulp used to use have an auto_retry decorator on mongoDB queries that was used everywhere and could result in duplicate records. Collections that have been converted to mongoengine in Pulp 2.7.0+ do not have the auto_retry decorator behavior at all currently.

The auto_retry behavior should be able to be enabled/disabled by the user. This story introduces a boolean 'unsafe_autoretry' setting in the the [database] section of server.conf which enabled/disables the use of auto_retry on both mongoengine Documents and non-mongoengine collections. The parameter will default to False. If set to True, all mongoDB queries will use an auto_retry decorator. Both mongoengine Documents and non-mongoengine Collections need to use the same retry decorator so that its behavior can be kept consistent.

To effectively decorate mongoengine it needs to be done during the init of each mongoengine model. This would cause a lot of duplication so it's recommended to have all mongoengine objects that inherit directly from Document inherit from a new class you will create called AutoRetryDocument. AutoRetryDocument should contain the init that does the decorating and takes *args and **args similar to:

class AutoRetryDocument(Document):

    _decorated_methods = (<figure out which methods to decorate>)

    def __init__(self, *args, **kwargs):
        super(AutoRetryDocument, self).__init__(*args, **kwargs)
        # do the decorating here similar to https://github.com/pulp/pulp/blob/d17e9e5c3fe2a3a2b81ef252304984c40003b7b7/server/pulp/server/db/connection.py#L210
Actions #1

Updated by rbarlow about 9 years ago

  • Tracker changed from Story to Task
Actions #2

Updated by rbarlow about 9 years ago

  • Project changed from 22 to Pulp
Actions #3

Updated by bmbouter about 9 years ago

  • Subject changed from [Task] Test mongoengine connection with no auto-retry and an HA mongoengine configuration to Test mongoengine connection with no auto-retry and an HA mongoengine configuration
Actions #4

Updated by bmbouter about 9 years ago

  • Description updated (diff)
Actions #5

Updated by bmbouter about 9 years ago

  • Description updated (diff)
Actions #6

Updated by bcourt about 9 years ago

  • Priority changed from Normal to High
Actions #7

Updated by bmbouter about 9 years ago

  • Tags Sprint Candidate added
Actions #8

Updated by mhrivnak about 9 years ago

  • Priority changed from High to Normal
Actions #9

Updated by bmbouter about 9 years ago

  • Description updated (diff)
Actions #10

Updated by rbarlow about 9 years ago

  • Description updated (diff)
Actions #11

Updated by rbarlow about 9 years ago

  • Description updated (diff)
Actions #12

Updated by bmbouter about 9 years ago

  • Tags Groomed added
Actions #13

Updated by bmbouter almost 9 years ago

  • Description updated (diff)
Actions #14

Updated by bmbouter almost 9 years ago

  • Groomed set to Yes
  • Tags deleted (Groomed)
Actions #15

Updated by bmbouter almost 9 years ago

  • Sprint Candidate set to Yes
  • Tags deleted (Sprint Candidate)
Actions #16

Updated by rbarlow almost 9 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to rbarlow
Actions #17

Updated by rbarlow almost 9 years ago

  • Tags Easy Fix added

Brian told me this would be super easy…

Actions #18

Updated by dkliban@redhat.com almost 9 years ago

  • Parent issue set to #1014
Actions #19

Updated by rbarlow almost 9 years ago

  • Blocked by Issue #1012: If the first database seed listed in server.conf is not the current Primary replica, Pulp will not be able to write to the database added
Actions #20

Updated by rbarlow almost 9 years ago

  • Status changed from ASSIGNED to NEW
  • Assignee deleted (rbarlow)
Actions #21

Updated by bmbouter over 8 years ago

  • Tracker changed from Task to Story
  • Subject changed from Test mongoengine connection with no auto-retry and an HA mongoengine configuration to As a user, I can have Pulp attempt use auto_retry application wide using the 'unsafe_autoretry' parameter
  • Description updated (diff)
  • Groomed changed from Yes to No
  • Tags deleted (Easy Fix)
Actions #22

Updated by bmbouter over 8 years ago

  • Description updated (diff)
Actions #23

Updated by bmbouter over 8 years ago

  • Blocked by deleted (Issue #1012: If the first database seed listed in server.conf is not the current Primary replica, Pulp will not be able to write to the database)
Actions #24

Updated by bmbouter over 8 years ago

  • Platform Release set to 2.7.0
Actions #25

Updated by amacdona@redhat.com over 8 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to amacdona@redhat.com
Actions #28

Updated by amacdona@redhat.com over 8 years ago

  • Status changed from ASSIGNED to POST

Added by Austin Macdonald over 8 years ago

Revision 805211fb | View on GitHub

Introduce unsafe_autoretry setting

re #1

Added by Austin Macdonald over 8 years ago

Revision 805211fb | View on GitHub

Introduce unsafe_autoretry setting

re #1

Actions #29

Updated by amacdona@redhat.com over 8 years ago

  • Status changed from POST to MODIFIED
Actions #32

Updated by dkliban@redhat.com over 8 years ago

  • Status changed from MODIFIED to 5
Actions #33

Updated by rbarlow about 8 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #34

Updated by bmbouter almost 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF