Issue #2822
closedUnicodeDecodeError in migration 0037
Description
It looks like we fixed this in the importers/yum/utils.py codepath for https://pulp.plan.io/issues/2093, but the migrations/0037_rpm_primary_repodata_new_location.py codepath still doesn't account for this issue. This can, in some instances, cause an exception during migrations
Halting migrations due to a migration failure.
'ascii' codec can't decode byte 0x9c in position 1: ordinal not in range(128)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 227, in main
return _auto_manage_db(options)
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 294, in _auto_manage_db
migrate_database(options)
File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 129, in migrate_database
update_current_version=not options.test)
File "/usr/lib/python2.7/site-packages/pulp/server/db/migrate/models.py", line 186, in apply_migration
migration.migrate()
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/migrations/0037_rpm_primary_repodata_new_location.py", line 44, in migrate
migrate_rpm_base(rpm_collection, rpm)
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/migrations/0037_rpm_primary_repodata_new_location.py", line 59, in migrate_rpm_base
delta['repodata'] = fix_location(unit['repodata'], unit['filename'])
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/migrations/0037_rpm_primary_repodata_new_location.py", line 73, in fix_location
faked_primary = fake_xml_element(repodata['primary'])
File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/migrations/0037_rpm_primary_repodata_new_location.py", line 111, in fake_xml_element
repodata_snippet.encode(codec)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x9c in position 1: ordinal not in range(128)
Updated by pcreech almost 7 years ago
- Status changed from NEW to CLOSED - NOTABUG
This is a downstream specific bug
Updated by dkliban@redhat.com over 6 years ago
- Status changed from CLOSED - NOTABUG to NEW
sudoSamurai in #pulp reported this issue when upgrading to 2.14.3. Looks like the migration needs to contain this block: https://github.com/pulp/pulp_rpm/blob/207085eb15e7e70419dac5441de97e2f98bd5fee/plugins/pulp_rpm/plugins/importers/yum/utils.py#L139
Updated by ttereshc over 6 years ago
- Status changed from NEW to CLOSED - NOTABUG
Bytes 0x9c in position 1
indicate that the repodata is not decompressed.
Compressed repodata was introduced in migration 39 but the error is seen in migration 37, so my only guess is that code has been upgraded (and thus newly synced/uploaded RPMs contain compressed repodata), while migrations hasn't been run yet.
It's very likely that to get to this state, pulp-manage-db
hasn't been run right after upgrade:
- Pulp < 2.12 is used
- stop Pulp services
- upgrade Pulp at least to the latest 2.12
- forget to run pulp-manage-db
- start back Pulp services
- sync or upload RPMs
- stop Pulp services and run pulp-manage-db
It's not possible to revert changes made by sync or upload.
Restore from backup, upgrade and don't forget to run pulp-manage-db
immediately after upgrade, before starting back Pulp services.