Pulp

Issue #4055

Updated by ttereshc about 6 years ago

"The migration for modulemd data":https://github.com/pulp/pulp_rpm/blob/2-master/plugins/pulp_rpm/plugins/migrations/0043_add_modulemd_modulemd-defaults.py introduces a new collection in the DB but doesn't create indexes before migrating a content.  
 So it's possible to have duplicated modulemd units in the DB.  
 Users would run into a problem during with the next upgrade because pulp-manage-db will try to build missing indexes. 

 <pre> 
 E11000 duplicate key error collection: pulp_database.units_modulemd index: name_1_stream_1_version_1_context_1_arch_1 dup key: { : "django", : "1.6", : 20180307130104, : "c2c572ec", : "noarch" } 
 Traceback (most recent call last): 
   File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 236, in main 
     return _auto_manage_db(options) 
   File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 303, in _auto_manage_db 
     migrate_database(options) 
   File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 148, in migrate_database 
     ensure_database_indexes() 
   File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 186, in ensure_database_indexes 
     model_class.ensure_indexes() 
   File "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 738, in ensure_indexes 
     collection = cls._get_collection() 
   File "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 210, in _get_collection 
     cls.ensure_indexes() 
   File "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 766, in ensure_indexes 
     collection.create_index(fields, background=background, **opts) 
   File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 1380, in create_index 
     self.__create_index(keys, kwargs) 
   File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 1290, in __create_index 
     sock_info, cmd, read_preference=ReadPreference.PRIMARY) 
   File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 205, in _command 
     read_concern=read_concern) 
   File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 211, in command 
     read_concern) 
   File "/usr/lib64/python2.7/site-packages/pymongo/network.py", line 100, in command 
     helpers._check_command_response(response_doc, msg, allowable_errors) 
   File "/usr/lib64/python2.7/site-packages/pymongo/helpers.py", line 189, in _check_command_response 
     raise DuplicateKeyError(errmsg, code, response) 
 DuplicateKeyError: E11000 duplicate key error collection: pulp_database.units_modulemd index: name_1_stream_1_version_1_context_1_arch_1 dup key: { : "django", : "1.6", : 20180307130104, : "c2c572ec", : "noarch" } 
 </pre> 

 The migration should build indexes before migrating content. 
 The migration should handle duplicates properly. 
 See example of how a similar case was handled "here":https://github.com/pulp/pulp_rpm/blob/2-master/plugins/pulp_rpm/plugins/migrations/0040_errata_pkglist_collection.py#L27.

Back