Migration Plugin

here's an example of my pulp_2to3_migration_pulp2repository table after doing this 4 times

pulpcore=# select pulp2_repo_id, pulp2_object_id, pulp3_repository_version_id from pulp_2to3_migration_pulp2repository;

      pulp2_repo_id      |     pulp2_object_id      |     pulp3_repository_version_id      
-------------------------+--------------------------+--------------------------------------
 pulp-uuid-rhel_6_x86_64 | 5ed68583b379890777e5ff86 | 
 pulp-uuid-rhel_6_x86_64 | 5ed68468b3798907780787d2 | 
 pulp-uuid-rhel_6_x86_64 | 5ed6819bb379890777e5ff5f | 
 pulp-uuid-rhel_6_x86_64 | 5ed67a84b379890776663e9c | 7de05591-f9e8-41f3-8cb1-c98524a39ce6

Note that this had https://github.com/pulp/pulp-2to3-migration/pull/152/ applied! Which could be related.

Here's the migration plan i used.

{
   "plugins":[
      {
         "type":"rpm",
         "repositories":[
            {
               "name":"pulp-uuid-rhel_6_x86_64",
               "repository_versions":[
                  {
                     "pulp2_repository_id":"pulp-uuid-rhel_6_x86_64",
                     "pulp2_distributor_repository_ids":[
                        "pulp-uuid-rhel_6_x86_64"
                     ]
                  }
               ],
               "pulp2_importer_repository_id":"pulp-uuid-rhel_6_x86_64"
            }                                 
         ]
      }
   ]
}

So to recap, to reproduce this:

1. Create a yum repo in pulp 2 with id 'pulp-uuid-rhel_6_x86_64'
2. Sync it and publish it
3. Run the above migration plan
4. Delete the repo
5. repeat steps 1,2,3

After re-running the migration plan, the 'migrated repo' with 'no_in_plan' set to false, will NOT have a version_href or distribution hrefs associated. The old one will

Reopening, as there ended up being an issue!

I'm still seeing this issue:

 DETAIL:  Key (base_path)=(Default_Organization/dev/My_Files) already exists.
 The above exception was the direct cause of the following exception:
 Traceback (most recent call last):
 File "/usr/lib/python3.6/site-packages/rq/worker.py", line 886, in perform_job
 rv = job.perform()
 File "/usr/lib/python3.6/site-packages/rq/job.py", line 664, in perform
 self._result = self._execute()
 File "/usr/lib/python3.6/site-packages/rq/job.py", line 670, in _execute
 return self.func(*self.args, **self.kwargs)
 File "/usr/local/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 233, in complex_repo_migration
 migrated_repo.pulp3_repository_version
 File "/usr/local/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 332, in migrate_repo_distribut
 pulp2dist, repo_version)
 File "/usr/local/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/iso/repository.py", line 79, in migrate_to_
 defaults=base_config)
 File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
 return getattr(self.get_queryset(), name)(*args, **kwargs)
 File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 559, in update_or_create
 obj, created = self._create_object_from_params(kwargs, params, lock=True)
 File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 583, in _create_object_from_params
 raise e
 File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 575, in _create_object_from_params
 obj = self.create(**params)
 File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 422, in create
 obj.save(force_insert=True, using=self.db)
 File "/usr/local/lib/python3.6/site-packages/pulpcore/app/models/base.py", line 110, in save
 return super().save(*args, **kwargs)
 File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 741, in save
 force_update=force_update, update_fields=update_fields)
 File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 776, in save_base
 parent_inserted = self._save_parents(cls, using, update_fields)
 File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 807, in _save_parents
 force_insert=parent_inserted,
 File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 870, in _save_table
 result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
 File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 908, in _do_insert
 using=using, raw=raw)
 File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
 return getattr(self.get_queryset(), name)(*args, **kwargs)
 File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
 return query.get_compiler(using=using).execute_sql(return_id)
 File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1375, in execute_sql
 cursor.execute(sql, params)
 File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
 return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
 File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
 return executor(sql, params, many, context)
 File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
 return self.cursor.execute(sql, params)
 File "/usr/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
 raise dj_exc_value.with_traceback(traceback) from exc_value
 File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
 return self.cursor.execute(sql, params)
 django.db.utils.IntegrityError: duplicate key value violates unique constraint "core_basedistribution_base_path_key"
 DETAIL:  Key (base_path)=(Default_Organization/dev/My_Files) already exists.

This was after i reinstalled the migration plugin from master. In this case the migration plan is a bit more complex:

{
   "plugins":[
      {
         "type":"iso",
         "repositories":[
            {
               "name":"Default_Organization-Cabinet-My_Files",
               "repository_versions":[
                  {
                     "pulp2_repository_id":"Default_Organization-Cabinet-My_Files",
                     "pulp2_distributor_repository_ids":[
                        "Default_Organization-Cabinet-My_Files"
                     ]
                  }
               ],
               "pulp2_importer_repository_id":"Default_Organization-Cabinet-My_Files"
            },
            {
               "name":"Published Library and dev view-700973453-909058347",
               "repository_versions":[
                  {
                     "pulp2_repository_id":"Default_Organization-1_0-Cabinet-My_Files",
                     "pulp2_distributor_repository_ids":[
                        "Default_Organization-1_0-Cabinet-My_Files",
                        "Default_Organization-Cabinet-My_Files_Dev"
                     ]
                  }
               ]
            }
         ]
      }
   ]
}

In this case it looks like the distributor 'Default_Organization-Cabinet-My_Files' is being migrated correctly, but none of the other two are.

Justin, i opened a separate issue per comment #10 https://pulp.plan.io/issues/6932

Applied in changeset pulp-2to3-migration|210bd49adc6254a099ead250290948fc1f4ffdf3.

Applied in changeset pulp:pulp-2to3-migrate|210bd49adc6254a099ead250290948fc1f4ffdf3.