Migration Plugin

Running the docker migrator more than once blows up on relating manifest blobs. It doesn't matter what the migration plan is specifically as long as the repos (/ repo content) overlaps.

Tested with these plans:

http POST :24817/pulp/api/v3/migration-plans/ plan='{"plugins": [{"type": "docker"}]}'

http POST :24817/pulp/api/v3/migration-plans/ plan='{"plugins": [{"type": "docker", "repositories": [{"name": "test-fixture-1", "repository_versions": [{"pulp2_repository_id": "test-fixture-1", "pulp2_distributor_repository_ids": ["test-fixture-1"]}]}]}]}'

Results:

Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: future: <Task finished coro=<DockerMigrator.migrate_content_to_pulp3() done, defined at /home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/plugin/docker/migrator.py:73> exception=IntegrityError('duplicate key value violates unique constraint "container_blobmanifest_manifest_id_manifest_blo_05da3e8d_uniq"\nDETAIL:  Key (manifest_id, manifest_blob_id)=(c20c1b65-83a0-4044-9f28-1e06ee2084fc, f03fbe62-6b09-48af-b53e-59d439a01313) already exists.\n',)>
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: Traceback (most recent call last):
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: return self.cursor.execute(sql, params)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "container_blobmanifest_manifest_id_manifest_blo_05da3e8d_uniq"
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: DETAIL:  Key (manifest_id, manifest_blob_id)=(c20c1b65-83a0-4044-9f28-1e06ee2084fc, f03fbe62-6b09-48af-b53e-59d439a01313) already exists.
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: The above exception was the direct cause of the following exception:
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: Traceback (most recent call last):
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/plugin/docker/migrator.py", line 80, in migrate_content_to_pulp3
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: await dm.create()
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/plugin/content.py", line 83, in create
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: await pipeline
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/api.py", line 209, in create_pipeline
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: await asyncio.gather(*futures)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/api.py", line 43, in __call__
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: await self.run()
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/plugin/docker/migrator.py", line 178, in run
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: self.relate_blob(dc)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/home/vagrant/devel/pulp-2to3-migration/pulp_2to3_migration/app/plugin/docker/migrator.py", line 214, in relate_blob
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: thru.save()
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 741, in save
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: force_update=force_update, update_fields=update_fields)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 779, in save_base
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: force_update, using, update_fields,
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 870, in _save_table
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/base.py", line 908, in _do_insert
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: using=using, raw=raw)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: return query.get_compiler(using=using).execute_sql(return_id)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1368, in execute_sql
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: cursor.execute(sql, params)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: return executor(sql, params, many, context)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: return self.cursor.execute(sql, params)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: raise dj_exc_value.with_traceback(traceback) from exc_value
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: return self.cursor.execute(sql, params)
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: django.db.utils.IntegrityError: duplicate key value violates unique constraint "container_blobmanifest_manifest_id_manifest_blo_05da3e8d_uniq"
Feb 04 20:18:08 pulp2-nightly-pulp3-source-centos7.localhost.example.com rq[11164]: DETAIL:  Key (manifest_id, manifest_blob_id)=(c20c1b65-83a0-4044-9f28-1e06ee2084fc, f03fbe62-6b09-48af-b53e-59d439a01313) already exists.