Project

Profile

Help

Issue #4055

Updated by ttereshc over 2 years ago

"The migration for modulemd data":https://github.com/pulp/pulp_rpm/blob/2-master/plugins/pulp_rpm/plugins/migrations/0043_add_modulemd_modulemd-defaults.py introduces a new collection in the DB but doesn't create indexes before migrating a content.
So it's possible to have duplicated modulemd units in the DB.
Users would run into a problem during with the next upgrade because pulp-manage-db will try to build missing indexes.

<pre>
E11000 duplicate key error collection: pulp_database.units_modulemd index: name_1_stream_1_version_1_context_1_arch_1 dup key: { : "django", : "1.6", : 20180307130104, : "c2c572ec", : "noarch" }
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 236, in main
return _auto_manage_db(options)
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 303, in _auto_manage_db
migrate_database(options)
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 148, in migrate_database
ensure_database_indexes()
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 186, in ensure_database_indexes
model_class.ensure_indexes()
File "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 738, in ensure_indexes
collection = cls._get_collection()
File "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 210, in _get_collection
cls.ensure_indexes()
File "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 766, in ensure_indexes
collection.create_index(fields, background=background, **opts)
File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 1380, in create_index
self.__create_index(keys, kwargs)
File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 1290, in __create_index
sock_info, cmd, read_preference=ReadPreference.PRIMARY)
File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 205, in _command
read_concern=read_concern)
File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 211, in command
read_concern)
File "/usr/lib64/python2.7/site-packages/pymongo/network.py", line 100, in command
helpers._check_command_response(response_doc, msg, allowable_errors)
File "/usr/lib64/python2.7/site-packages/pymongo/helpers.py", line 189, in _check_command_response
raise DuplicateKeyError(errmsg, code, response)
DuplicateKeyError: E11000 duplicate key error collection: pulp_database.units_modulemd index: name_1_stream_1_version_1_context_1_arch_1 dup key: { : "django", : "1.6", : 20180307130104, : "c2c572ec", : "noarch" }
</pre>

The migration should build indexes before migrating content.
The migration should handle duplicates properly.
See example of how a similar case was handled "here":https://github.com/pulp/pulp_rpm/blob/2-master/plugins/pulp_rpm/plugins/migrations/0040_errata_pkglist_collection.py#L27.

Back