Project

Profile

Help

Story #1

Updated by bmbouter over 8 years ago

Pulp used to use have an "auto_retry decorator":https://github.com/pulp/pulp/blob/d17e9e5c3fe2a3a2b81ef252304984c40003b7b7/server/pulp/server/db/connection.py#L163-L189 on mongoDB queries that was used everywhere and could result in duplicate records. Collections that have been converted to mongoengine in Pulp 2.7.0+ do not have the auto_retry decorator behavior at all currently. 

 The auto_retry behavior should be able to be enabled/disabled by the user. This story introduces a boolean 'unsafe_autoretry' setting in the the [database] section of server.conf which enabled/disables the use of auto_retry on both mongoengine Documents and non-mongoengine collections. The parameter will default to False. If set to True, all mongoDB queries will use an auto_retry decorator. Both mongoengine Documents and non-mongoengine Collections need to use the same retry decorator so that its behavior can be kept consistent. 

 To effectively decorate mongoengine it needs to be done during the __init__ of each mongoengine model. This would cause a lot of duplication so it's recommended to have all mongoengine objects that inherit directly from Document inherit from a new class you will create called AutoRetryDocument. AutoRetryDocument should contain the __init__ that does the decorating and takes *args and **args similar to: 

 <pre> 
 class AutoRetryDocument(Document): 

     _decorated_methods = (<figure out which methods to decorate>) 

     def __init__(self, *args, **kwargs): 
         super(AutoRetryDocument, self).__init__(*args, **kwargs) 
         # do the decorating here similar to https://github.com/pulp/pulp/blob/d17e9e5c3fe2a3a2b81ef252304984c40003b7b7/server/pulp/server/db/connection.py#L210 https://github.com/pulp/pulp/blob/d17e9e5c3fe2a3a2b81ef252304984c40003b7b7/server/pulp/server/db/connection.py#L200-L205 
 </pre>

Back