Project

Profile

Help

Issue #7889

Handle migration when same distributor is being re-used for different repositories in between the plans

Added by ipanova@redhat.com about 2 months ago. Updated 13 days ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Create plan and run:


{
    "plan": {
        "plugins": [
            {
                "repositories": [
                    {
                        "name": "file",
                        "pulp2_importer_repository_id": "file",
                        "repository_versions": [
                            {
                                "pulp2_distributor_repository_ids": [
                                    "file2"
                                ],
                                "pulp2_repository_id": "file"
                            }
                        ]
                    },
                    {
                        "name": "file2",
                        "pulp2_importer_repository_id": "file2",
                        "repository_versions": [
                            {
                                "pulp2_distributor_repository_ids": [
                                    "file-many"
                                ],
                                "pulp2_repository_id": "file2"
                            }
                        ]
                    }
                ],
                "type": "iso"
            }
        ]
    },

Create another plan and run:

{
    "plan": {
        "plugins": [
            {
                "repositories": [
                    {
                        "name": "file",
                        "pulp2_importer_repository_id": "file",
                        "repository_versions": [
                            {
                                "pulp2_distributor_repository_ids": [
                                    "file-many"
                                ],
                                "pulp2_repository_id": "file"
                            }
                        ]
                    },
                    {
                        "name": "file2",
                        "pulp2_importer_repository_id": "file2",
                        "repository_versions": [
                            {
                                "pulp2_distributor_repository_ids": [
                                    "file-large"
                                ],
                                "pulp2_repository_id": "file2"
                            }
                        ]
                    }
                ],
                "type": "iso"
            }
        ]
    },

File2 repo will have 2 distributions displayed file-large and file-many. In addition there is a concern of hitting base_path collision because same distributor file-many has been used to distribute 2 different repos.

NOTE Importers will suffer as well from correctness problem in case same importer will be used for 2 different repos in between the migration plans

History

#1 Updated by ipanova@redhat.com about 2 months ago

  • Description updated (diff)

#2 Updated by ttereshc about 2 months ago

Arguably simpler situation to grasp the problem, is to imagine when two distributors are being swapped.

First migration plan/run:
repo A with dist A
repo B with dist B

Second migration plan/run:
repo A with dist B
repo B with dist A

#3 Updated by ttereshc about 2 months ago

One idea would be to put relations from migration plan in a table and use it to identify old/new data.

class RepoSetup:
    repo_id = TextField()
    importer_id = TextField()
    distributor_id = TextField()
    status = IntegerField(choices=models.IntegerChoices('Status', 'old up_to_date new').choices)
    
    class Meta:
        unique_together = (
            (repo_id, distributor_id),
    )

On each migration run:

  • before pre-migration, set status for all rows to old.
  • then create records with status new or update with status up_to_date. (FWIW< this would likely be a per triplet operation and not a bulk one)
  • use this data to clean up old relations between premigrated repos and premigrated importers/distributors and set is_migrated=false for the distributors which changed the repository they are distributing.
  • at pre-migration time, use this table to check whether anything needs to be done for the repo version setup or if it can be skipped.

Outside of the scope of this ticket: this model can also be used as one of conditions to identify no-to-few changes.

#4 Updated by ipanova@redhat.com 13 days ago

Before I forget again, posting here the rough idea how to deal with the problem ( one of the discussed approaches).

Use repo_version_setup() instead of get_importers_repos/get_distributors_repo(). This will require to get rid of simple migration plan first. https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/pre_migration.py#L361

Please register to edit this issue

Also available in: Atom PDF