Project

Profile

Help

Task #5845

closed

Story #3821: As a user, I can migrate from Pulp 2 to Pulp 3

Update get_pulp3_repository_setup so repos are grouped by plugin type

Added by ipanova@redhat.com almost 5 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Motivation:

Repositories became typed, therefore for each incoming repo we need to identify it's type in order to migrate it
https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/migration.py#L68

for pulp2repo in pulp2repos_qs:
                pulp3_repo_name = pulp2repo.pulp2_repo_id
                repository_class = PLUGIN_MIGRATORS.get(pulp2repo.type).pulp3_repository
                repo, created = repository_class.objects.get_or_create(
                    name=pulp3_repo_name,
                    description=pulp2repo.pulp2_description)

Identifying `repository_class` for each repo might have performance impact

In case we'd like to 'migrate' a repo to pulp3 that is not present in pulp2, or in other words - migrate pulp2 repo with new name in pulp3 we'd have troubles to find its `repository_class`

for pulp3_repo_name in repos_to_create:
                try:
                    pulp2repo = pulp2repos_qs.get(pulp2_repo_id=pulp3_repo_name)
                except Pulp2Repository.DoesNotExist:
                    description = pulp3_repo_name
                else:
                    description = pulp2repo.pulp2_description

                repository_class = PLUGIN_MIGRATORS.get(pulp2repo.type).pulp3_repository
                repo, created = repository_class.objects.get_or_create(
                    name=pulp3_repo_name,
                    description=description)

https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/app/migration.py#L91

Solution

Having repos grouped by plugin type( iso, docker. rpm) would help to deal with above mentioned cases.

Update plan.get_pulp3_repository_setup() so it would have following structure:

{iso: {repoA : repo_data, repoB: repo_data}, docker: {repoA : repo_data, repoB: repo_data}}
Actions #1

Updated by fao89 almost 5 years ago

  • Tracker changed from Issue to Task
  • % Done set to 0
  • Tags Pulp 2 to 3 Migrate added
Actions #2

Updated by ipanova@redhat.com almost 5 years ago

  • Parent issue set to #3821
Actions #3

Updated by dalley almost 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley
Actions #4

Updated by dalley almost 5 years ago

  • Status changed from ASSIGNED to POST

https://github.com/pulp/pulp-2to3-migration/pull/66

In my PR I ended up doing something slightly different that I think will end up being much cleaner. Instead of getting a single "pulp3_repo_setup" structure which is discriminated by type, I modified all of the parsing code so that it creates many "plugin migration plans" which you acquire using the method "plan.get_plugin_plans()".

Each of those plugin migration plan objects keep track of all their {repositories to create/repositories/importers/distributors} separately instead of merging them all into one big list (although I kept the methods to get the complete lists). I noticed that we were looking up the "migrators" a lot, so the new object also makes the associated "migrator" (if it exists) an attribute, as well as whether the plan is "empty". That helped to clean up the code a lot and will hopefully allow it to be cleaned up more later.

Added by dalley almost 5 years ago

Revision 2dd9da11 | View on GitHub

Refactor migration plan parsing to keep track of plugin types

closes: #5845 https://pulp.plan.io/issues/5845 closes: #5899 https://pulp.plan.io/issues/5899

Added by dalley almost 5 years ago

Revision 2dd9da11 | View on GitHub

Refactor migration plan parsing to keep track of plugin types

closes: #5845 https://pulp.plan.io/issues/5845 closes: #5899 https://pulp.plan.io/issues/5899

Added by dalley almost 5 years ago

Revision 2dd9da11 | View on GitHub

Refactor migration plan parsing to keep track of plugin types

closes: #5845 https://pulp.plan.io/issues/5845 closes: #5899 https://pulp.plan.io/issues/5899

Actions #5

Updated by dalley almost 5 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #6

Updated by ttereshc almost 5 years ago

  • Project changed from Pulp to Migration Plugin
Actions #7

Updated by ttereshc over 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
  • Sprint/Milestone set to 0.1.0
Actions #8

Updated by bmbouter over 4 years ago

  • Tags deleted (Pulp 2 to 3 Migrate)

We're removing the 'Pulp 2 to 3 Migrate' tag to allow membership of the Migration Plugin project in Redmine to be the controlling area https://pulp.plan.io/projects/migration

Also available in: Atom PDF