Project

Profile

Help

Story #4871

When changing unit_key_fields, the relevant DB units may need to be prepared during migration BEFORE indexes are applied

Added by quba42 about 1 year ago. Updated about 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.20.0
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Sprint 53

Description

pulp-manage-db currently creates indexes before migrations are run.
In most cases this is desirable behaviour since it ensures Mongo will enforce uniqueness constraints (see https://pulp.plan.io/issues/4055).

However, when dropping a field from the unit_key_fields of a DB model, this means that the new key is applied (as an index) BEFORE the migration has had a chance to prepare the existing units for the new key.
This can easily turn previously unique units into duplicates, thus breaking any attempt to migrate them.
This makes it essentially impossible to change a unit key in this way.

Associated revisions

Revision 2ebcb7bf View on GitHub
Added by quba42 about 1 year ago

Add an optional pre index part for DB migrations

This is needed to provide a way to prepare the data base for an index change BEFORE that index is applied. The new mechanism is subject to certain risks and limitations. It should only be used where absolutely necessary.

ref #4055, #4138 https://pulp.plan.io/issues/4055 https://pulp.plan.io/issues/4138

closes #4871 https://pulp.plan.io/issues/4871

History

#1 Updated by quba42 about 1 year ago

The following change (https://github.com/pulp/pulp/pull/3919) is designed to tackle this issue, albeit with some significant limitations and not without introducing some risk.

On the positive side, it does so in a way that should not affect any existing migrations that do not make use of the newly introduced "prepare_reindex_migration" at all.

The hope is that usage of this new function will remain a rare exception. Appropriate warnings and usage instructions will still be added to all doc strings.

#2 Updated by quba42 about 1 year ago

Note that this whole issue was first revealed by the fix for https://pulp.plan.io/issues/4138
Namely: https://github.com/pulp/pulp_deb/pull/83

#3 Updated by ppicka about 1 year ago

  • Project changed from Debian Support to Pulp

#4 Updated by CodeHeeler about 1 year ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 53

#5 Updated by ttereshc about 1 year ago

  • Tracker changed from Issue to Story
  • % Done set to 0
  • Platform Release set to 2.20.0
  • Groomed changed from No to Yes
  • Sprint Candidate changed from No to Yes

#6 Updated by ttereshc about 1 year ago

  • Status changed from NEW to POST
  • Assignee set to quba42

#7 Updated by quba42 about 1 year ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#8 Updated by ttereshc about 1 year ago

  • Sprint/Milestone set to 2.20.0

#9 Updated by ttereshc about 1 year ago

  • Status changed from MODIFIED to 5

#10 Updated by ttereshc about 1 year ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF