https://pulp.plan.io/https://pulp.plan.io/favicon.ico2020-06-02T17:20:23ZPulpMigration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=575082020-06-02T17:20:23Zjsherril@redhat.comjsherril@redhat.com
<ul></ul><p>here's an example of my pulp_2to3_migration_pulp2repository table after doing this 4 times</p>
<p>pulpcore=# select pulp2_repo_id, pulp2_object_id, pulp3_repository_version_id from pulp_2to3_migration_pulp2repository;</p>
<pre><code> pulp2_repo_id | pulp2_object_id | pulp3_repository_version_id
-------------------------+--------------------------+--------------------------------------
pulp-uuid-rhel_6_x86_64 | 5ed68583b379890777e5ff86 |
pulp-uuid-rhel_6_x86_64 | 5ed68468b3798907780787d2 |
pulp-uuid-rhel_6_x86_64 | 5ed6819bb379890777e5ff5f |
pulp-uuid-rhel_6_x86_64 | 5ed67a84b379890776663e9c | 7de05591-f9e8-41f3-8cb1-c98524a39ce6
</code></pre> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=575172020-06-02T17:38:46Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>ASSIGNED</i></li><li><strong>Assignee</strong> set to <i>ipanova@redhat.com</i></li><li><strong>Sprint</strong> set to <i>Sprint 74</i></li></ul> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=575212020-06-02T18:36:10Zjsherril@redhat.comjsherril@redhat.com
<ul></ul><p>Note that this had <a href="https://github.com/pulp/pulp-2to3-migration/pull/152/" class="external">https://github.com/pulp/pulp-2to3-migration/pull/152/</a> applied! Which could be related.</p> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=575672020-06-03T15:18:53Zjsherril@redhat.comjsherril@redhat.com
<ul></ul><p>Here's the migration plan i used.</p>
<pre><code>{
"plugins":[
{
"type":"rpm",
"repositories":[
{
"name":"pulp-uuid-rhel_6_x86_64",
"repository_versions":[
{
"pulp2_repository_id":"pulp-uuid-rhel_6_x86_64",
"pulp2_distributor_repository_ids":[
"pulp-uuid-rhel_6_x86_64"
]
}
],
"pulp2_importer_repository_id":"pulp-uuid-rhel_6_x86_64"
}
]
}
]
}
</code></pre>
<p>So to recap, to reproduce this:</p>
<ol>
<li>Create a yum repo in pulp 2 with id 'pulp-uuid-rhel_6_x86_64'</li>
<li>Sync it and publish it</li>
<li>Run the above migration plan</li>
<li>Delete the repo</li>
<li>repeat steps 1,2,3</li>
</ol>
<p>After re-running the migration plan, the 'migrated repo' with 'no_in_plan' set to false, will NOT have a version_href or distribution hrefs associated. The old one will</p> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=576012020-06-04T02:01:15Zjsherril@redhat.comjsherril@redhat.com
<ul><li><strong>Status</strong> changed from <i>ASSIGNED</i> to <i>CLOSED - NOTABUG</i></li></ul><p>This was user error, the integration tests were not waiting for the task group to complete!!</p> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=576202020-06-04T12:18:25Zjsherril@redhat.comjsherril@redhat.com
<ul></ul><p>Reopening, as there ended up being an issue!</p> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=576232020-06-04T12:27:25Zjsherril@redhat.comjsherril@redhat.com
<ul><li><strong>Status</strong> changed from <i>CLOSED - NOTABUG</i> to <i>NEW</i></li></ul> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=576242020-06-04T13:09:12Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>POST</i></li><li><strong>Triaged</strong> changed from <i>No</i> to <i>Yes</i></li></ul> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=576402020-06-04T14:17:56Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Status</strong> changed from <i>POST</i> to <i>MODIFIED</i></li></ul><p>Applied in changeset <a class="changeset" title="Handle already-migrated 're-created' pulp2 repos closes #6887 https://pulp.plan.io/issues/6887" href="https://pulp.plan.io/projects/pulp/repository/pulp-2to3-migration/revisions/210bd49adc6254a099ead250290948fc1f4ffdf3">pulp:pulp-2to3-migration|210bd49adc6254a099ead250290948fc1f4ffdf3</a>.</p>
<p><a href="https://github.com/pulp/pulp-2to3-migration/pull/157" class="external">https://github.com/pulp/pulp-2to3-migration/pull/157</a></p> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=576512020-06-04T17:14:57Zjsherril@redhat.comjsherril@redhat.com
<ul></ul><p>I'm still seeing this issue:</p>
<pre><code> DETAIL: Key (base_path)=(Default_Organization/dev/My_Files) already exists.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/rq/worker.py", line 886, in perform_job
rv = job.perform()
File "/usr/lib/python3.6/site-packages/rq/job.py", line 664, in perform
self._result = self._execute()
File "/usr/lib/python3.6/site-packages/rq/job.py", line 670, in _execute
return self.func(*self.args, **self.kwargs)
File "/usr/local/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 233, in complex_repo_migration
migrated_repo.pulp3_repository_version
File "/usr/local/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 332, in migrate_repo_distribut
pulp2dist, repo_version)
File "/usr/local/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/iso/repository.py", line 79, in migrate_to_
defaults=base_config)
File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 559, in update_or_create
obj, created = self._create_object_from_params(kwargs, params, lock=True)
File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 583, in _create_object_from_params
raise e
File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 575, in _create_object_from_params
obj = self.create(**params)
File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 422, in create
obj.save(force_insert=True, using=self.db)
File "/usr/local/lib/python3.6/site-packages/pulpcore/app/models/base.py", line 110, in save
return super().save(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 741, in save
force_update=force_update, update_fields=update_fields)
File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 776, in save_base
parent_inserted = self._save_parents(cls, using, update_fields)
File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 807, in _save_parents
force_insert=parent_inserted,
File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 870, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File "/usr/lib/python3.6/site-packages/django/db/models/base.py", line 908, in _do_insert
using=using, raw=raw)
File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1375, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "core_basedistribution_base_path_key"
DETAIL: Key (base_path)=(Default_Organization/dev/My_Files) already exists.
</code></pre>
<p>This was after i reinstalled the migration plugin from master. In this case the migration plan is a bit more complex:</p>
<pre><code>{
"plugins":[
{
"type":"iso",
"repositories":[
{
"name":"Default_Organization-Cabinet-My_Files",
"repository_versions":[
{
"pulp2_repository_id":"Default_Organization-Cabinet-My_Files",
"pulp2_distributor_repository_ids":[
"Default_Organization-Cabinet-My_Files"
]
}
],
"pulp2_importer_repository_id":"Default_Organization-Cabinet-My_Files"
},
{
"name":"Published Library and dev view-700973453-909058347",
"repository_versions":[
{
"pulp2_repository_id":"Default_Organization-1_0-Cabinet-My_Files",
"pulp2_distributor_repository_ids":[
"Default_Organization-1_0-Cabinet-My_Files",
"Default_Organization-Cabinet-My_Files_Dev"
]
}
]
}
]
}
]
}
</code></pre>
<p>In this case it looks like the distributor 'Default_Organization-Cabinet-My_Files' is being migrated correctly, but none of the other two are.</p> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=578122020-06-09T09:07:11Zipanova@redhat.comipanova@redhat.com
<ul></ul><p>Justin, i opened a separate issue per comment #10 <a href="https://pulp.plan.io/issues/6932" class="external">https://pulp.plan.io/issues/6932</a></p> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=579262020-06-11T12:01:33Zipanova@redhat.comipanova@redhat.com
<ul></ul><p>Applied in changeset <a class="changeset" title="Handle already-migrated 're-created' pulp2 repos closes #6887 https://pulp.plan.io/issues/6887" href="https://pulp.plan.io/projects/migration/repository/pulp-2to3-migration/revisions/210bd49adc6254a099ead250290948fc1f4ffdf3">pulp-2to3-migration|210bd49adc6254a099ead250290948fc1f4ffdf3</a>.</p> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=590842020-07-05T00:32:56Zipanova@redhat.comipanova@redhat.com
<ul></ul><p>Applied in changeset <a class="changeset" title="Handle already-migrated 're-created' pulp2 repos closes #6887 https://pulp.plan.io/issues/6887" href="https://pulp.plan.io/projects/pulp/repository/pulp-2to3-migrate/revisions/210bd49adc6254a099ead250290948fc1f4ffdf3">pulp:pulp-2to3-migrate|210bd49adc6254a099ead250290948fc1f4ffdf3</a>.</p> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=611712020-08-20T16:38:43Zttereshcttereshc@redhat.com
<ul><li><strong>Sprint/Milestone</strong> set to <i>0.2.0</i></li></ul> Migration Plugin - Issue #6887: migration doesn't seem to handle already-migrated 're-created' pulp2 reposhttps://pulp.plan.io/issues/6887?journal_id=612182020-08-20T18:06:34Zttereshcttereshc@redhat.com
<ul><li><strong>Status</strong> changed from <i>MODIFIED</i> to <i>CLOSED - CURRENTRELEASE</i></li></ul>