Actions
Issue #2037
closed
Migration failing after 2.9 upgrade
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
2.9.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 4
Quarter:
Description
Migration failure on upgrade from 2.8->2.9
Steps
1. Setup 2.8 with the following repos
pulp-admin rpm repo create --repo-id rhel5.11 --feed http://[content-internal]/content/dist/rhel/rhui/server/5/5.11/x86_64/os/ --download-policy on_demand
pulp-admin rpm repo sync run --repo-id rhel5.11
Upgrade to 2.9
[ ~]# sudo -u apache pulp-manage-db
Attempting to connect to localhost:27017
Attempting to connect to localhost:27017
Write concern for Mongo connection: {}
/usr/lib/python2.6/site-packages/pulp/server/db/connection.py:159: DeprecationWarning: add_son_manipulator is deprecated
_DATABASE.add_son_manipulator(NamespaceInjector())
Loading content types.
Loading type descriptors []
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Loading unit model: python_package = pulp_python.plugins.models:Package
Loading unit model: docker_blob = pulp_docker.plugins.models:Blob
Loading unit model: docker_manifest = pulp_docker.plugins.models:Manifest
Loading unit model: docker_image = pulp_docker.plugins.models:Image
Loading unit model: docker_tag = pulp_docker.plugins.models:Tag
Loading unit model: erratum = pulp_rpm.plugins.db.models:Errata
Loading unit model: distribution = pulp_rpm.plugins.db.models:Distribution
Loading unit model: srpm = pulp_rpm.plugins.db.models:SRPM
Loading unit model: package_group = pulp_rpm.plugins.db.models:PackageGroup
Loading unit model: package_category = pulp_rpm.plugins.db.models:PackageCategory
Loading unit model: iso = pulp_rpm.plugins.db.models:ISO
Loading unit model: package_environment = pulp_rpm.plugins.db.models:PackageEnvironment
Loading unit model: drpm = pulp_rpm.plugins.db.models:DRPM
Loading unit model: package_langpacks = pulp_rpm.plugins.db.models:PackageLangpacks
Loading unit model: rpm = pulp_rpm.plugins.db.models:RPM
Loading unit model: yum_repo_metadata_file = pulp_rpm.plugins.db.models:YumMetadataFile
Loading unit model: puppet_module = pulp_puppet.plugins.db.models:Module
Updating the database with types []
/usr/lib/python2.6/site-packages/pulp/server/db/model/base.py:96: DeprecationWarning: ensure_index is deprecated. Use create_index instead.
unique=unique, background=True)
Found the following type definitions that were not present in the update collection [puppet_module, docker_tag, docker_manifest, python_package, erratum, yum_repo_metadata_file, package_group, docker_blob, package_category, iso, package_environment, drpm, srpm, rpm, distribution, docker_image]
Updating the database with types [puppet_module, drpm, package_langpacks, docker_manifest, python_package, erratum, yum_repo_metadata_file, package_group, docker_blob, package_category, iso, package_environment, docker_tag, distribution, rpm, srpm, docker_image]
/usr/lib/python2.6/site-packages/pulp/plugins/types/database.py:277: DeprecationWarning: save is deprecated. Use insert_one or replace_one instead
content_type_collection.save(content_type)
Content types loaded.
Ensuring the admin role and user are in place.
Admin role and user are in place.
Beginning database migrations.
Applying pulp.server.db.migrations version 24
/usr/lib/python2.6/site-packages/pulp/server/db/migrations/0024_distributor_schema_change.py:23: DeprecationWarning: save is deprecated. Use insert_one or replace_one instead
collection.save(distributor)
/usr/lib/python2.6/site-packages/mongoengine/document.py:367: DeprecationWarning: update is deprecated. Use replace_one, update_one or update_many instead.
upsert=upsert, **write_concern)
Migration to pulp.server.db.migrations version 24 complete.
Migration package pulp_docker.plugins.migrations is up to date at version 2
Migration package pulp_puppet.plugins.migrations is up to date at version 5
Migration package pulp_python.plugins.migrations is up to date at version 1
Applying pulp_rpm.plugins.migrations version 33
Applying migration pulp_rpm.plugins.migrations.0033_checksums_and_templates failed.
Halting migrations due to a migration failure.
not well-formed (invalid token): line 6, column 9
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 193, in main
return _auto_manage_db(options)
File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 256, in _auto_manage_db
migrate_database(options)
File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 124, in migrate_database
update_current_version=not options.test)
File "/usr/lib/python2.6/site-packages/pulp/server/db/migrate/models.py", line 186, in apply_migration
migration.migrate()
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 45, in migrate
migrate_rpm_base(rpm_collection, rpm)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 76, in migrate_rpm_base
delta['repodata'] = _modify_xml(unit['repodata'])
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 101, in _modify_xml
faked_filelists = fake_xml_element(repodata['filelists'])
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 159, in fake_xml_element
return ET.fromstring(fake_xml.encode(codec))
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 963, in XML
parser.feed(text)
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 1245, in feed
self._parser.Parse(data, 0)
ExpatError: not well-formed (invalid token): line 6, column 9
From pulp.log
Jun 27 14:13:19 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:INFO: Applying pulp_rpm.plugins.migrations version 33
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) Applying migration pulp_rpm.plugins.migrations.0033_checksums_and_templates failed.
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) Halting migrations due to a migration failure.
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: not well-formed (invalid token): line 6, column 9
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) Traceback (most recent call last):
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 193, in main
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) return _auto_manage_db(options)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 256, in _auto_manage_db
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) migrate_database(options)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 124, in migrate_database
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) update_current_version=not options.test)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp/server/db/migrate/models.py", line 186, in apply_migration
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) migration.migrate()
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 45, in migrate
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) migrate_rpm_base(rpm_collection, rpm)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 76, in migrate_rpm_base
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) delta['repodata'] = _modify_xml(unit['repodata'])
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 101, in _modify_xml
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) faked_filelists = fake_xml_element(repodata['filelists'])
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 159, in fake_xml_element
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) return ET.fromstring(fake_xml.encode(codec))
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 963, in XML
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) parser.feed(text)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 1245, in feed
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) self._parser.Parse(data, 0)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) ExpatError: not well-formed (invalid token): line 6, column 9
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256)
[root@ibm-x3650m4-01-vm-12 ~]#
Files
Actions
.
"other" and "filelists" XML snippet processing fixed to not do slicing
These snippets were mistakenly being sent through the "remove_fake_element" function, which would slice them incorrectly and shave off a single character. This was ok most of the time, because it would be a white space character. But occasionally it was the final ">" of the XML.
fixes #2037 https://pulp.plan.io/issues/2037