Story #4871
closed
When changing unit_key_fields, the relevant DB units may need to be prepared during migration BEFORE indexes are applied
100%
Description
pulp-manage-db currently creates indexes before migrations are run.
In most cases this is desirable behaviour since it ensures Mongo will enforce uniqueness constraints (see https://pulp.plan.io/issues/4055).
However, when dropping a field from the unit_key_fields of a DB model, this means that the new key is applied (as an index) BEFORE the migration has had a chance to prepare the existing units for the new key.
This can easily turn previously unique units into duplicates, thus breaking any attempt to migrate them.
This makes it essentially impossible to change a unit key in this way.
Updated by quba42 almost 5 years ago
The following change (https://github.com/pulp/pulp/pull/3919) is designed to tackle this issue, albeit with some significant limitations and not without introducing some risk.
On the positive side, it does so in a way that should not affect any existing migrations that do not make use of the newly introduced "prepare_reindex_migration" at all.
The hope is that usage of this new function will remain a rare exception. Appropriate warnings and usage instructions will still be added to all doc strings.
Updated by quba42 almost 5 years ago
Note that this whole issue was first revealed by the fix for https://pulp.plan.io/issues/4138
Namely: https://github.com/pulp/pulp_deb/pull/83
Updated by ppicka almost 5 years ago
- Project changed from Debian Support to Pulp
Added by quba42 almost 5 years ago
Updated by CodeHeeler almost 5 years ago
- Triaged changed from No to Yes
- Sprint set to Sprint 53
Updated by ttereshc almost 5 years ago
- Tracker changed from Issue to Story
- % Done set to 0
- Platform Release set to 2.20.0
- Groomed changed from No to Yes
- Sprint Candidate changed from No to Yes
Updated by ttereshc almost 5 years ago
- Status changed from NEW to POST
- Assignee set to quba42
Updated by quba42 almost 5 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulp|2ebcb7bf26d22c023cdb85fd95744997e91b3b76.
Updated by ttereshc almost 5 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
.
Add an optional pre index part for DB migrations
This is needed to provide a way to prepare the data base for an index change BEFORE that index is applied. The new mechanism is subject to certain risks and limitations. It should only be used where absolutely necessary.
ref #4055, #4138 https://pulp.plan.io/issues/4055 https://pulp.plan.io/issues/4138
closes #4871 https://pulp.plan.io/issues/4871