https://pulp.plan.io/https://pulp.plan.io/favicon.ico2019-04-01T19:18:37ZPulpRPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=353192019-04-01T19:18:37Zdkliban@redhat.com
<ul></ul><p>Looks like the problem was with some of the content on the CDN. Here is a patch that fixes the problem. We'll include this fix in 2.19.1.</p>
<pre><code>diff --git a/plugins/pulp_rpm/plugins/migrations/0043_add_modulemd_modulemd-defaults.py b/plugins/pulp_rpm/plugins/migrations/0043_add_modulemd_modulemd-defaults.py
index 7736a5a8..bb2bcb55 100644
--- a/plugins/pulp_rpm/plugins/migrations/0043_add_modulemd_modulemd-defaults.py
+++ b/plugins/pulp_rpm/plugins/migrations/0043_add_modulemd_modulemd-defaults.py
@@ -336,14 +336,26 @@ def migrate(*args, **kwargs):
repos_to_republish = set()
for file in metadatafiles:
repository = Repository.objects.get(repo_id=file.repo_id)
- with gzip.open(file._storage_path, 'r') as fp:
- working_dir = "/var/cache/pulp"
- modulemds, defaults = load(fp, working_dir)
- add_modulemds(repository, modulemds, working_dir)
- add_defaults(repository, defaults, working_dir)
- repository_controller.disassociate_units(repository, [file])
- repository_controller.rebuild_content_unit_counts(repository)
- repos_to_republish.add(file.repo_id)
+ try:
+ with gzip.open(file._storage_path, 'r') as fp:
+ working_dir = "/var/cache/pulp"
+ modulemds, defaults = load(fp, working_dir)
+ add_modulemds(repository, modulemds, working_dir)
+ add_defaults(repository, defaults, working_dir)
+ repository_controller.disassociate_units(repository, [file])
+ repository_controller.rebuild_content_unit_counts(repository)
+ repos_to_republish.add(file.repo_id)
+ except IOError:
+ _logger.info(file.unit_key)
+ with open(file._storage_path, 'r') as fp:
+ working_dir = "/var/cache/pulp"
+ modulemds, defaults = load(fp, working_dir)
+ add_modulemds(repository, modulemds, working_dir)
+ add_defaults(repository, defaults, working_dir)
+ repository_controller.disassociate_units(repository, [file])
+ repository_controller.rebuild_content_unit_counts(repository)
+ repos_to_republish.add(file.repo_id)
+
if repos_to_republish:
with open('/var/lib/pulp/0043_add_modulemd_modulemd-defaults.txt', 'w') as f:
f.write(str(list(repos_to_republish)))
</code></pre> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=353212019-04-01T19:28:09Zttereshcttereshc@redhat.com
<ul><li><strong>Project</strong> changed from <i>31</i> to <i>RPM Support</i></li><li><strong>Subject</strong> changed from <i>Pulp failed upgrade on production 2.16.3 --> 2.18.1 (IO Error: not a gzipped file)</i> to <i>Migration of modular content can fail (upgrade from <=2.16 to 2.17+)</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/35321/diff?detail_id=36060">diff</a>)</li></ul> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=353252019-04-01T19:28:57Zttereshcttereshc@redhat.com
<ul><li><strong>Has duplicate</strong> <i><a class="issue tracker-1 status-12 priority-7 priority-high2 closed" href="/issues/4619">Issue #4619</a>: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)</i> added</li></ul> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=353272019-04-01T20:24:58Zkfiresmithkfiresmith@gmail.com
<ul></ul><p>Thanks very much DKliban. I'm back in business. For what it's worth for the rest of Pulp 2.x's lifespan, trapping exceptions as done in the patch with some logging goodness to stdout at bail-time would be very much welcome to all those who may encounter such a traceback.</p> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=353492019-04-02T13:13:04Zdkliban@redhat.com
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>ASSIGNED</i></li><li><strong>Assignee</strong> set to <i>dkliban@redhat.com</i></li></ul> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=353522019-04-02T13:58:06Zdkliban@redhat.com
<ul><li><strong>Status</strong> changed from <i>ASSIGNED</i> to <i>MODIFIED</i></li></ul><p>Applied in changeset <a class="changeset" title="Problem: migration fails for uncompressed modules files Solution: handle IOError and read the un..." href="https://pulp.plan.io/projects/pulp_rpm/repository/9/revisions/4e4a6b4ad4514df16f07ce43fd2e197695f4cbf5">4e4a6b4ad4514df16f07ce43fd2e197695f4cbf5</a>.</p> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=357292019-04-12T09:56:00Zttereshcttereshc@redhat.com
<ul><li><strong>Sprint</strong> set to <i>Sprint 51</i></li></ul> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=375822019-04-15T19:57:54Zbmbouterbmbouter@redhat.com
<ul><li><strong>Tags</strong> <i>Pulp 2</i> added</li></ul> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=410952019-04-23T19:44:00Zbherring
<ul><li><strong>Copied to</strong> <i><a class="issue tracker-5 status-9 priority-7 priority-high2 closed" href="/issues/4729">Test #4729</a>: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)</i> added</li></ul> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=433082019-05-02T12:54:59Zdkliban@redhat.com
<ul><li><strong>Platform Release</strong> set to <i>2.19.1</i></li></ul> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=434842019-05-03T19:33:16Zdkliban@redhat.com
<ul><li><strong>Sprint/Milestone</strong> set to <i>2.19.1</i></li></ul> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=439512019-05-21T12:15:17Zdkliban@redhat.com
<ul></ul><p>Applied in changeset <a class="changeset" title="Problem: migration fails for uncompressed modules files Solution: handle IOError and read the un..." href="https://pulp.plan.io/projects/pulp_rpm/repository/9/revisions/dc9cb3bee2f183b80ea35bff5084957bc1be93fa">dc9cb3bee2f183b80ea35bff5084957bc1be93fa</a>.</p> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=440182019-05-22T19:26:16Zdkliban@redhat.com
<ul><li><strong>Status</strong> changed from <i>MODIFIED</i> to <i>5</i></li></ul> RPM Support - Issue #4617: Migration of modular content can fail (upgrade from <=2.16 to 2.17+)https://pulp.plan.io/issues/4617?journal_id=442222019-05-30T16:42:13Zdkliban@redhat.com
<ul><li><strong>Status</strong> changed from <i>5</i> to <i>CLOSED - CURRENTRELEASE</i></li></ul>