Project

Profile

Help

Story #4871

When changing unit_key_fields, the relevant DB units may need to be prepared during migration BEFORE indexes are applied

Added by quba42 8 months ago. Updated 6 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Platform Release:
2.20.0
Blocks Release:
Backwards Incompatible:
No
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 53

Description

pulp-manage-db currently creates indexes before migrations are run.
In most cases this is desirable behaviour since it ensures Mongo will enforce uniqueness constraints (see https://pulp.plan.io/issues/4055).

However, when dropping a field from the unit_key_fields of a DB model, this means that the new key is applied (as an index) BEFORE the migration has had a chance to prepare the existing units for the new key.
This can easily turn previously unique units into duplicates, thus breaking any attempt to migrate them.
This makes it essentially impossible to change a unit key in this way.

Associated revisions

Revision 2ebcb7bf View on GitHub
Added by quba42 8 months ago

Add an optional pre index part for DB migrations

This is needed to provide a way to prepare the data base for an index
change BEFORE that index is applied. The new mechanism is subject to
certain risks and limitations. It should only be used where absolutely
necessary.

ref #4055, #4138
https://pulp.plan.io/issues/4055
https://pulp.plan.io/issues/4138

closes #4871
https://pulp.plan.io/issues/4871

History

#1 Updated by quba42 8 months ago

The following change (https://github.com/pulp/pulp/pull/3919) is designed to tackle this issue, albeit with some significant limitations and not without introducing some risk.

On the positive side, it does so in a way that should not affect any existing migrations that do not make use of the newly introduced "prepare_reindex_migration" at all.

The hope is that usage of this new function will remain a rare exception. Appropriate warnings and usage instructions will still be added to all doc strings.

#2 Updated by quba42 8 months ago

Note that this whole issue was first revealed by the fix for https://pulp.plan.io/issues/4138
Namely: https://github.com/pulp/pulp_deb/pull/83

#3 Updated by ppicka 8 months ago

  • Project changed from Debian Support to Pulp

#4 Updated by CodeHeeler 8 months ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 53

#5 Updated by ttereshc 8 months ago

  • Tracker changed from Issue to Story
  • % Done set to 0
  • Platform Release set to 2.20.0
  • Groomed changed from No to Yes
  • Sprint Candidate changed from No to Yes

#6 Updated by ttereshc 8 months ago

  • Status changed from NEW to POST
  • Assignee set to quba42

#7 Updated by quba42 8 months ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#8 Updated by ttereshc 7 months ago

  • Sprint/Milestone set to 2.20.0

#9 Updated by ttereshc 7 months ago

  • Status changed from MODIFIED to ON_QA

#10 Updated by ttereshc 6 months ago

  • Status changed from ON_QA to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF