Story #1
closed
Task #1014: Short Term Improvements for Pulp's use of MongoDB
As a user, I can have Pulp attempt use auto_retry application wide using the 'unsafe_autoretry' parameter
0%
Description
Pulp used to use have an auto_retry decorator on mongoDB queries that was used everywhere and could result in duplicate records. Collections that have been converted to mongoengine in Pulp 2.7.0+ do not have the auto_retry decorator behavior at all currently.
The auto_retry behavior should be able to be enabled/disabled by the user. This story introduces a boolean 'unsafe_autoretry' setting in the the [database] section of server.conf which enabled/disables the use of auto_retry on both mongoengine Documents and non-mongoengine collections. The parameter will default to False. If set to True, all mongoDB queries will use an auto_retry decorator. Both mongoengine Documents and non-mongoengine Collections need to use the same retry decorator so that its behavior can be kept consistent.
To effectively decorate mongoengine it needs to be done during the init of each mongoengine model. This would cause a lot of duplication so it's recommended to have all mongoengine objects that inherit directly from Document inherit from a new class you will create called AutoRetryDocument. AutoRetryDocument should contain the init that does the decorating and takes *args and **args similar to:
class AutoRetryDocument(Document):
_decorated_methods = (<figure out which methods to decorate>)
def __init__(self, *args, **kwargs):
super(AutoRetryDocument, self).__init__(*args, **kwargs)
# do the decorating here similar to https://github.com/pulp/pulp/blob/d17e9e5c3fe2a3a2b81ef252304984c40003b7b7/server/pulp/server/db/connection.py#L210
Updated by bmbouter almost 10 years ago
- Subject changed from [Task] Test mongoengine connection with no auto-retry and an HA mongoengine configuration to Test mongoengine connection with no auto-retry and an HA mongoengine configuration
Updated by bmbouter over 9 years ago
- Sprint Candidate set to Yes
- Tags deleted (
Sprint Candidate)
Updated by rbarlow over 9 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to rbarlow
Updated by rbarlow over 9 years ago
- Blocked by Issue #1012: If the first database seed listed in server.conf is not the current Primary replica, Pulp will not be able to write to the database added
Updated by rbarlow over 9 years ago
- Status changed from ASSIGNED to NEW
- Assignee deleted (
rbarlow)
Updated by bmbouter over 9 years ago
- Tracker changed from Task to Story
- Subject changed from Test mongoengine connection with no auto-retry and an HA mongoengine configuration to As a user, I can have Pulp attempt use auto_retry application wide using the 'unsafe_autoretry' parameter
- Description updated (diff)
- Groomed changed from Yes to No
- Tags deleted (
Easy Fix)
Updated by bmbouter over 9 years ago
- Blocked by deleted (Issue #1012: If the first database seed listed in server.conf is not the current Primary replica, Pulp will not be able to write to the database)
Updated by amacdona@redhat.com over 9 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to amacdona@redhat.com
Updated by amacdona@redhat.com over 9 years ago
- Status changed from ASSIGNED to POST
Added by Austin Macdonald over 9 years ago
Added by Austin Macdonald over 9 years ago
Revision 805211fb | View on GitHub
Introduce unsafe_autoretry setting
re #1
Updated by amacdona@redhat.com over 9 years ago
- Status changed from POST to MODIFIED
Updated by dkliban@redhat.com about 9 years ago
- Status changed from MODIFIED to 5
Updated by rbarlow almost 9 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
.
Introduce unsafe_autoretry setting
re #1