https://pulp.plan.io/https://pulp.plan.io/favicon.ico2016-05-25T16:57:36ZPulpPulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=117292016-05-25T16:57:36Zkfiresmithkfiresmith@gmail.com
<ul></ul><p>@jortel is aware of the issue and is looking into the migration unit code.</p> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=117342016-05-25T17:36:57Zbmbouterbmbouter@redhat.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/11734/diff?detail_id=12481">diff</a>)</li></ul> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=117362016-05-25T18:12:47Zrbarlow
<ul><li><strong>Priority</strong> changed from <i>Normal</i> to <i>Urgent</i></li></ul><p>I suggest urgent priority since this issue blocks users from upgrading to 2.8.3, which is a security release.</p> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=117372016-05-25T18:15:16Zbmbouterbmbouter@redhat.com
<ul></ul><p>+1 to urgent, and if this is being worked on it should be in ASSIGNED state.</p> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=117382016-05-25T19:00:29Zjortel@redhat.comjortel@redhat.com
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>ASSIGNED</i></li><li><strong>Assignee</strong> set to <i>jortel@redhat.com</i></li></ul> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=117472016-05-26T13:04:57Zehelms@redhat.comehelms@redhat.com
<ul></ul><p>Related to this migration another upgrade traceback, if you'd like a separate issue for it please let me know.</p>
<pre><code>
Migration to pulp_docker.plugins.migrations version 2 complete.
Applying pulp_puppet.plugins.migrations version 4
Migration to pulp_puppet.plugins.migrations version 4 complete.
Applying pulp_puppet.plugins.migrations version 5
***Note. You may want to re-publish the list of repos found in /tmp/repos_to_republish.txt.
This migration fixed an issue with modules installation related to wrong puppet_module name.
Migration to pulp_puppet.plugins.migrations version 5 complete.
Applying pulp_rpm.plugins.migrations version 28
Applying migration pulp_rpm.plugins.migrations.0028_standard_storage_path failed.
Halting migrations due to a migration failure.
'variant'
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 193, in main
return _auto_manage_db(options)
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 256, in _auto_manage_db
migrate_database(options)
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 124, in migrate_database
update_current_version=not options.test)
File "/usr/lib/python2.7/site-packages/pulp/server/db/migrate/models.py", line 186, in apply_migration
migration.migrate()
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/migrations/0028_standard_storage_path.py", line 17, in migrate
migration()
File "/usr/lib/python2.7/site-packages/pulp/plugins/migration/standard_storage_path.py", line 262, in __call__
for unit in chain(*self.plans):
File "/usr/lib/python2.7/site-packages/pulp/plugins/migration/standard_storage_path.py", line 188, in __iter__
unit.new_path = self._new_path(unit)
File "/usr/lib/python2.7/site-packages/pulp/plugins/migration/standard_storage_path.py", line 165, in _new_path
digest = unit.key_digest()
File "/usr/lib/python2.7/site-packages/pulp/plugins/migration/standard_storage_path.py", line 338, in key_digest
for key, value in sorted(self.key.items()):
File "/usr/lib/python2.7/site-packages/pulp/plugins/migration/standard_storage_path.py", line 319, in key
return dict([(k, self.document[k]) for k in self.plan.key_fields])
KeyError: 'variant'
</code></pre> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=117482016-05-26T13:18:28Zkfiresmithkfiresmith@gmail.com
<ul></ul><p>Hi Eric,<br>
That actually looks unrelated; we think what I'm seeing is an issue manifesting from outside the DB, and your traceback looks very much like an issue from within the database. Just my $0.02 but I think your issue will turn out to be something independent of what I reported. :)</p> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=117752016-05-26T19:15:42Zjortel@redhat.comjortel@redhat.com
<ul></ul><p>Reproduced with tons of help from <a class="user active" href="https://pulp.plan.io/users/240">kfiresmith</a>. Big thanks.</p>
<p>Some of the plugins released with early 2.8 had been updated (incorrectly) to use the new platform storage model (introduced in 2.8) for unit files. This resulted in a period of time when units were created with storage paths that should have ended with the file names but did not. Apparently the ISO importer was one of these plugins but was fixed in later 2.8 releases. The ISO storage-path migration just needs to be updated to detect these units and correct the storage path.</p> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=117892016-05-26T20:41:46Zjortel@redhat.comjortel@redhat.com
<ul><li><strong>Status</strong> changed from <i>ASSIGNED</i> to <i>POST</i></li></ul><p><a href="https://github.com/pulp/pulp_rpm/pull/882" class="external">https://github.com/pulp/pulp_rpm/pull/882</a></p> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=118042016-05-27T14:43:21Zbmbouterbmbouter@redhat.com
<ul><li><strong>Triaged</strong> changed from <i>No</i> to <i>Yes</i></li></ul> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=119002016-05-31T12:51:23Zkfiresmithkfiresmith@gmail.com
<ul></ul><p>Thanks very much Jeff - I can confirm this fixed the failed migration we saw.</p> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=119042016-05-31T14:53:06Zjortel@redhat.comjortel@redhat.com
<ul></ul><p>PR updated to deal with missing <em>variant</em> in distributions.</p> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=119212016-05-31T16:58:05Zjortel@redhat.comjortel@redhat.com
<ul><li><strong>Status</strong> changed from <i>POST</i> to <i>MODIFIED</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>Applied in changeset <a class="changeset" title="Fix storage path migration issue with ISO and distributions. closes #1945" href="https://pulp.plan.io/projects/pulp_rpm/repository/9/revisions/bb0b21e2b900c1200d60341973b8f4b446fdd484">pulp_rpm:bb0b21e2b900c1200d60341973b8f4b446fdd484</a>.</p> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=119332016-05-31T16:59:16Zsemyerssean.myers@redhat.com
<ul><li><strong>Platform Release</strong> set to <i>2.8.4</i></li></ul> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=119752016-06-01T14:31:27Zsemyerssean.myers@redhat.com
<ul><li><strong>Status</strong> changed from <i>MODIFIED</i> to <i>5</i></li></ul> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=119912016-06-01T17:29:56Zpthomas@redhat.com
<ul><li><strong>Status</strong> changed from <i>5</i> to <i>6</i></li></ul><p>verified<br>
Ran migration on an upgraded pulp with data in rpm.puppet, iso and docker repos</p>
<pre><code>[root@mgmt8 ~]# sudo -u apache pulp-manage-db
Attempting to connect to localhost:27017
Attempting to connect to localhost:27017
Write concern for Mongo connection: {}
Loading content types.
Loading type descriptors []
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Loading unit model: docker_blob = pulp_docker.plugins.models:Blob
Loading unit model: docker_manifest = pulp_docker.plugins.models:Manifest
Loading unit model: docker_image = pulp_docker.plugins.models:Image
Loading unit model: docker_tag = pulp_docker.plugins.models:Tag
Loading unit model: puppet_module = pulp_puppet.plugins.db.models:Module
Loading unit model: erratum = pulp_rpm.plugins.db.models:Errata
Loading unit model: distribution = pulp_rpm.plugins.db.models:Distribution
Loading unit model: package_group = pulp_rpm.plugins.db.models:PackageGroup
Loading unit model: package_category = pulp_rpm.plugins.db.models:PackageCategory
Loading unit model: iso = pulp_rpm.plugins.db.models:ISO
Loading unit model: package_environment = pulp_rpm.plugins.db.models:PackageEnvironment
Loading unit model: drpm = pulp_rpm.plugins.db.models:DRPM
Loading unit model: srpm = pulp_rpm.plugins.db.models:SRPM
Loading unit model: rpm = pulp_rpm.plugins.db.models:RPM
Loading unit model: yum_repo_metadata_file = pulp_rpm.plugins.db.models:YumMetadataFile
Updating the database with types []
Found the following type definitions that were not present in the update collection [puppet_module, docker_tag, docker_manifest, docker_blob, erratum, yum_repo_metadata_file, package_group, package_category, iso, package_environment, drpm, srpm, rpm, distribution, docker_image]
Updating the database with types [puppet_module, drpm, erratum, docker_blob, docker_manifest, yum_repo_metadata_file, package_group, package_category, iso, package_environment, docker_tag, distribution, rpm, srpm, docker_image]
Content types loaded.
Ensuring the admin role and user are in place.
Admin role and user are in place.
Beginning database migrations.
Migration package pulp.server.db.migrations is up to date at version 23
Migration package pulp_docker.plugins.migrations is up to date at version 2
Migration package pulp_puppet.plugins.migrations is up to date at version 4
Applying pulp_rpm.plugins.migrations version 29
/usr/lib/python2.7/site-packages/pulp_rpm/plugins/migrations/0029_remove_rpm_unused_fields.py:23: DeprecationWarning: update is deprecated. Use replace_one, update_one or update_many instead.
multi=True
/usr/lib/python2.7/site-packages/pulp_rpm/plugins/migrations/0029_remove_rpm_unused_fields.py:28: DeprecationWarning: update is deprecated. Use replace_one, update_one or update_many instead.
multi=True
/usr/lib/python2.7/site-packages/mongoengine/document.py:367: DeprecationWarning: update is deprecated. Use replace_one, update_one or update_many instead.
upsert=upsert, **write_concern)
Migration to pulp_rpm.plugins.migrations version 29 complete.
Applying pulp_rpm.plugins.migrations version 30
/usr/lib/python2.7/site-packages/pulp_rpm/plugins/migrations/0030_remove_errata_unused_fields.py:23: DeprecationWarning: update is deprecated. Use replace_one, update_one or update_many instead.
multi=True
Migration to pulp_rpm.plugins.migrations version 30 complete.
Loading unit model: docker_blob = pulp_docker.plugins.models:Blob
Loading unit model: docker_manifest = pulp_docker.plugins.models:Manifest
Loading unit model: docker_image = pulp_docker.plugins.models:Image
Loading unit model: docker_tag = pulp_docker.plugins.models:Tag
Loading unit model: puppet_module = pulp_puppet.plugins.db.models:Module
Loading unit model: erratum = pulp_rpm.plugins.db.models:Errata
Loading unit model: distribution = pulp_rpm.plugins.db.models:Distribution
Loading unit model: package_group = pulp_rpm.plugins.db.models:PackageGroup
Loading unit model: package_category = pulp_rpm.plugins.db.models:PackageCategory
Loading unit model: iso = pulp_rpm.plugins.db.models:ISO
Loading unit model: package_environment = pulp_rpm.plugins.db.models:PackageEnvironment
Loading unit model: drpm = pulp_rpm.plugins.db.models:DRPM
Loading unit model: srpm = pulp_rpm.plugins.db.models:SRPM
Loading unit model: rpm = pulp_rpm.plugins.db.models:RPM
Loading unit model: yum_repo_metadata_file = pulp_rpm.plugins.db.models:YumMetadataFile
Database migrations complete.
[root@mgmt8 ~]#
</code></pre> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=120882016-06-06T18:54:28Zsemyerssean.myers@redhat.com
<ul><li><strong>Status</strong> changed from <i>6</i> to <i>CLOSED - CURRENTRELEASE</i></li></ul> Pulp - Issue #1945: pulp 2.8.3 migration fails on unit 0028https://pulp.plan.io/issues/1945?journal_id=388912019-04-15T20:29:55Zbmbouterbmbouter@redhat.com
<ul><li><strong>Tags</strong> <i>Pulp 2</i> added</li></ul>