Story #1
closed
Task #1014: Short Term Improvements for Pulp's use of MongoDB
As a user, I can have Pulp attempt use auto_retry application wide using the 'unsafe_autoretry' parameter
0%
Description
Pulp used to use have an auto_retry decorator on mongoDB queries that was used everywhere and could result in duplicate records. Collections that have been converted to mongoengine in Pulp 2.7.0+ do not have the auto_retry decorator behavior at all currently.
The auto_retry behavior should be able to be enabled/disabled by the user. This story introduces a boolean 'unsafe_autoretry' setting in the the [database] section of server.conf which enabled/disables the use of auto_retry on both mongoengine Documents and non-mongoengine collections. The parameter will default to False. If set to True, all mongoDB queries will use an auto_retry decorator. Both mongoengine Documents and non-mongoengine Collections need to use the same retry decorator so that its behavior can be kept consistent.
To effectively decorate mongoengine it needs to be done during the init of each mongoengine model. This would cause a lot of duplication so it's recommended to have all mongoengine objects that inherit directly from Document inherit from a new class you will create called AutoRetryDocument. AutoRetryDocument should contain the init that does the decorating and takes *args and **args similar to:
class AutoRetryDocument(Document):
_decorated_methods = (<figure out which methods to decorate>)
def __init__(self, *args, **kwargs):
super(AutoRetryDocument, self).__init__(*args, **kwargs)
# do the decorating here similar to https://github.com/pulp/pulp/blob/d17e9e5c3fe2a3a2b81ef252304984c40003b7b7/server/pulp/server/db/connection.py#L210
.
Introduce unsafe_autoretry setting
re #1