Issue #2037
closedMigration failing after 2.9 upgrade
Description
Migration failure on upgrade from 2.8->2.9
Steps
1. Setup 2.8 with the following repos
pulp-admin rpm repo create --repo-id rhel5.11 --feed http://[content-internal]/content/dist/rhel/rhui/server/5/5.11/x86_64/os/ --download-policy on_demand
pulp-admin rpm repo sync run --repo-id rhel5.11
Upgrade to 2.9
[ ~]# sudo -u apache pulp-manage-db
Attempting to connect to localhost:27017
Attempting to connect to localhost:27017
Write concern for Mongo connection: {}
/usr/lib/python2.6/site-packages/pulp/server/db/connection.py:159: DeprecationWarning: add_son_manipulator is deprecated
_DATABASE.add_son_manipulator(NamespaceInjector())
Loading content types.
Loading type descriptors []
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Loading unit model: python_package = pulp_python.plugins.models:Package
Loading unit model: docker_blob = pulp_docker.plugins.models:Blob
Loading unit model: docker_manifest = pulp_docker.plugins.models:Manifest
Loading unit model: docker_image = pulp_docker.plugins.models:Image
Loading unit model: docker_tag = pulp_docker.plugins.models:Tag
Loading unit model: erratum = pulp_rpm.plugins.db.models:Errata
Loading unit model: distribution = pulp_rpm.plugins.db.models:Distribution
Loading unit model: srpm = pulp_rpm.plugins.db.models:SRPM
Loading unit model: package_group = pulp_rpm.plugins.db.models:PackageGroup
Loading unit model: package_category = pulp_rpm.plugins.db.models:PackageCategory
Loading unit model: iso = pulp_rpm.plugins.db.models:ISO
Loading unit model: package_environment = pulp_rpm.plugins.db.models:PackageEnvironment
Loading unit model: drpm = pulp_rpm.plugins.db.models:DRPM
Loading unit model: package_langpacks = pulp_rpm.plugins.db.models:PackageLangpacks
Loading unit model: rpm = pulp_rpm.plugins.db.models:RPM
Loading unit model: yum_repo_metadata_file = pulp_rpm.plugins.db.models:YumMetadataFile
Loading unit model: puppet_module = pulp_puppet.plugins.db.models:Module
Updating the database with types []
/usr/lib/python2.6/site-packages/pulp/server/db/model/base.py:96: DeprecationWarning: ensure_index is deprecated. Use create_index instead.
unique=unique, background=True)
Found the following type definitions that were not present in the update collection [puppet_module, docker_tag, docker_manifest, python_package, erratum, yum_repo_metadata_file, package_group, docker_blob, package_category, iso, package_environment, drpm, srpm, rpm, distribution, docker_image]
Updating the database with types [puppet_module, drpm, package_langpacks, docker_manifest, python_package, erratum, yum_repo_metadata_file, package_group, docker_blob, package_category, iso, package_environment, docker_tag, distribution, rpm, srpm, docker_image]
/usr/lib/python2.6/site-packages/pulp/plugins/types/database.py:277: DeprecationWarning: save is deprecated. Use insert_one or replace_one instead
content_type_collection.save(content_type)
Content types loaded.
Ensuring the admin role and user are in place.
Admin role and user are in place.
Beginning database migrations.
Applying pulp.server.db.migrations version 24
/usr/lib/python2.6/site-packages/pulp/server/db/migrations/0024_distributor_schema_change.py:23: DeprecationWarning: save is deprecated. Use insert_one or replace_one instead
collection.save(distributor)
/usr/lib/python2.6/site-packages/mongoengine/document.py:367: DeprecationWarning: update is deprecated. Use replace_one, update_one or update_many instead.
upsert=upsert, **write_concern)
Migration to pulp.server.db.migrations version 24 complete.
Migration package pulp_docker.plugins.migrations is up to date at version 2
Migration package pulp_puppet.plugins.migrations is up to date at version 5
Migration package pulp_python.plugins.migrations is up to date at version 1
Applying pulp_rpm.plugins.migrations version 33
Applying migration pulp_rpm.plugins.migrations.0033_checksums_and_templates failed.
Halting migrations due to a migration failure.
not well-formed (invalid token): line 6, column 9
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 193, in main
return _auto_manage_db(options)
File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 256, in _auto_manage_db
migrate_database(options)
File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 124, in migrate_database
update_current_version=not options.test)
File "/usr/lib/python2.6/site-packages/pulp/server/db/migrate/models.py", line 186, in apply_migration
migration.migrate()
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 45, in migrate
migrate_rpm_base(rpm_collection, rpm)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 76, in migrate_rpm_base
delta['repodata'] = _modify_xml(unit['repodata'])
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 101, in _modify_xml
faked_filelists = fake_xml_element(repodata['filelists'])
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 159, in fake_xml_element
return ET.fromstring(fake_xml.encode(codec))
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 963, in XML
parser.feed(text)
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 1245, in feed
self._parser.Parse(data, 0)
ExpatError: not well-formed (invalid token): line 6, column 9
From pulp.log
Jun 27 14:13:19 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:INFO: Applying pulp_rpm.plugins.migrations version 33
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) Applying migration pulp_rpm.plugins.migrations.0033_checksums_and_templates failed.
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) Halting migrations due to a migration failure.
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: not well-formed (invalid token): line 6, column 9
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) Traceback (most recent call last):
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 193, in main
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) return _auto_manage_db(options)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 256, in _auto_manage_db
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) migrate_database(options)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 124, in migrate_database
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) update_current_version=not options.test)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp/server/db/migrate/models.py", line 186, in apply_migration
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) migration.migrate()
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 45, in migrate
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) migrate_rpm_base(rpm_collection, rpm)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 76, in migrate_rpm_base
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) delta['repodata'] = _modify_xml(unit['repodata'])
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 101, in _modify_xml
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) faked_filelists = fake_xml_element(repodata['filelists'])
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/migrations/0033_checksums_and_templates.py", line 159, in fake_xml_element
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) return ET.fromstring(fake_xml.encode(codec))
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 963, in XML
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) parser.feed(text)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 1245, in feed
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) self._parser.Parse(data, 0)
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256) ExpatError: not well-formed (invalid token): line 6, column 9
Jun 27 14:13:29 ibm-x3650m4-01-vm-12 pulp: pulp.server.db.manage:CRITICAL: (31855-72256)
[root@ibm-x3650m4-01-vm-12 ~]#
Files
Updated by mhrivnak over 8 years ago
What OS was this on, and which specific pulp versions did have you before and after the upgrade?
I was not able to reproduce this just now, going from the 2.8-dev branch to the 2.9-dev branch on F23.
Updated by pthomas@redhat.com over 8 years ago
I went from 2.8.5 beta to 2.9 beta
The repos that I had were
'errata repo with the feed https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/test_errata_install/
[root@ibm-x3650m4-01-vm-12 ~]# pulp-admin rpm repo list
+----------------------------------------------------------------------+
RPM Repositories
+----------------------------------------------------------------------+
Id: rhel5.11
Display Name: None
Description: None
Content Unit Counts:
Erratum: 3135
Package Category: 6
Package Group: 99
Rpm: 17117
Yum Repo Metadata File: 1
Id: errata
Display Name: None
Description: None
Content Unit Counts:
Distribution: 1
Erratum: 52
Package Category: 2
Package Group: 3
Rpm: 3
Id: errata2
Display Name: None
Description: None
Content Unit Counts:
Erratum: 4
Rpm: 11
Id: zoo
Display Name: None
Description: None
Content Unit Counts:
Erratum: 4
Package Category: 1
Package Group: 2
Rpm: 31
[root@ ~]# yum update --skip-broken
Loaded plugins: product-id, pulp-profile-update, search-disabled-repos, security, subscription-manager
Setting up Update Process
errata | 2.1 kB 00:00
errata2 | 2.1 kB 00:00
pulp | 3.6 kB 00:00
pulp/primary_db | 27 kB 00:00
zoo | 2.1 kB 00:00
zoo/primary | 3.7 kB 00:00
zoo 32/32
Resolving Dependencies
--> Running transaction check
---> Package grinder_test_package.noarch 0:4.0-1.fc14 will be updated
---> Package grinder_test_package.noarch 0:5.0-1.fc14 will be an update
---> Package pulp-admin-client.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-admin-client.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package pulp-agent.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-agent.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package pulp-consumer-client.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-consumer-client.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package pulp-puppet-admin-extensions.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-puppet-admin-extensions.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package pulp-puppet-consumer-extensions.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-puppet-consumer-extensions.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package pulp-puppet-handlers.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-puppet-handlers.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package pulp-puppet-plugins.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-puppet-plugins.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package pulp-rpm-admin-extensions.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-rpm-admin-extensions.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package pulp-rpm-consumer-extensions.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-rpm-consumer-extensions.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package pulp-rpm-handlers.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-rpm-handlers.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package pulp-rpm-plugins.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-rpm-plugins.noarch 0:2.9.0-0.2.beta.el6 will be an update
--> Processing Dependency: repoview for package: pulp-rpm-plugins-2.9.0-0.2.beta.el6.noarch
---> Package pulp-rpm-yumplugins.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-rpm-yumplugins.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package pulp-selinux.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-selinux.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package pulp-server.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package pulp-server.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package python-pulp-agent-lib.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package python-pulp-agent-lib.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package python-pulp-bindings.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package python-pulp-bindings.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package python-pulp-client-lib.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package python-pulp-client-lib.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package python-pulp-common.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package python-pulp-common.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package python-pulp-oid_validation.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package python-pulp-oid_validation.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package python-pulp-puppet-common.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package python-pulp-puppet-common.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package python-pulp-repoauth.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package python-pulp-repoauth.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package python-pulp-rpm-common.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package python-pulp-rpm-common.noarch 0:2.9.0-0.2.beta.el6 will be an update
---> Package python-pulp-streamer.noarch 0:2.8.5-0.1.beta.el6 will be updated
---> Package python-pulp-streamer.noarch 0:2.9.0-0.2.beta.el6 will be an update
--> Running transaction check
---> Package repoview.noarch 0:0.6.6-1.el6 will be installed
--> Processing Dependency: python-kid >= 0.6.3 for package: repoview-0.6.6-1.el6.noarch
--> Running transaction check
---> Package python-kid.noarch 0:0.9.6-5.1.el6 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=========================================================================================================
Package Arch Version Repository Size
=========================================================================================================
Updating:
grinder_test_package noarch 5.0-1.fc14 errata2 1.6 k
pulp-admin-client noarch 2.9.0-0.2.beta.el6 pulp 125 k
pulp-agent noarch 2.9.0-0.2.beta.el6 pulp 74 k
pulp-consumer-client noarch 2.9.0-0.2.beta.el6 pulp 88 k
pulp-puppet-admin-extensions noarch 2.9.0-0.2.beta.el6 pulp 51 k
pulp-puppet-consumer-extensions noarch 2.9.0-0.2.beta.el6 pulp 25 k
pulp-puppet-handlers noarch 2.9.0-0.2.beta.el6 pulp 30 k
pulp-puppet-plugins noarch 2.9.0-0.2.beta.el6 pulp 105 k
pulp-rpm-admin-extensions noarch 2.9.0-0.2.beta.el6 pulp 149 k
pulp-rpm-consumer-extensions noarch 2.9.0-0.2.beta.el6 pulp 48 k
pulp-rpm-handlers noarch 2.9.0-0.2.beta.el6 pulp 73 k
pulp-rpm-plugins noarch 2.9.0-0.2.beta.el6 pulp 329 k
pulp-rpm-yumplugins noarch 2.9.0-0.2.beta.el6 pulp 45 k
pulp-selinux noarch 2.9.0-0.2.beta.el6 pulp 82 k
pulp-server noarch 2.9.0-0.2.beta.el6 pulp 833 k
python-pulp-agent-lib noarch 2.9.0-0.2.beta.el6 pulp 91 k
python-pulp-bindings noarch 2.9.0-0.2.beta.el6 pulp 111 k
python-pulp-client-lib noarch 2.9.0-0.2.beta.el6 pulp 202 k
python-pulp-common noarch 2.9.0-0.2.beta.el6 pulp 123 k
python-pulp-oid_validation noarch 2.9.0-0.2.beta.el6 pulp 64 k
python-pulp-puppet-common noarch 2.9.0-0.2.beta.el6 pulp 32 k
python-pulp-repoauth noarch 2.9.0-0.2.beta.el6 pulp 77 k
python-pulp-rpm-common noarch 2.9.0-0.2.beta.el6 pulp 65 k
python-pulp-streamer noarch 2.9.0-0.2.beta.el6 pulp 74 k
Installing for dependencies:
python-kid noarch 0.9.6-5.1.el6 beaker-Server-optional 161 k
repoview noarch 0.6.6-1.el6 epel 40 k
Transaction Summary
=========================================================================================================
Install 2 Package(s)
Upgrade 24 Package(s)
Total download size: 3.0 M
Is this ok [y/N]: y
Downloading Packages:
(1/26): grinder_test_package-5.0-1.fc14.noarch.rpm | 1.6 kB 00:00
(2/26): pulp-admin-client-2.9.0-0.2.beta.el6.noarch.rpm | 125 kB 00:00
(3/26): pulp-agent-2.9.0-0.2.beta.el6.noarch.rpm | 74 kB 00:00
(4/26): pulp-consumer-client-2.9.0-0.2.beta.el6.noarch.rpm | 88 kB 00:00
(5/26): pulp-puppet-admin-extensions-2.9.0-0.2.beta.el6.noarch.rpm | 51 kB 00:00
(6/26): pulp-puppet-consumer-extensions-2.9.0-0.2.beta.el6.noarch.rpm | 25 kB 00:00
(7/26): pulp-puppet-handlers-2.9.0-0.2.beta.el6.noarch.rpm | 30 kB 00:00
(8/26): pulp-puppet-plugins-2.9.0-0.2.beta.el6.noarch.rpm | 105 kB 00:00
(9/26): pulp-rpm-admin-extensions-2.9.0-0.2.beta.el6.noarch.rpm | 149 kB 00:00
(10/26): pulp-rpm-consumer-extensions-2.9.0-0.2.beta.el6.noarch.rpm | 48 kB 00:00
(11/26): pulp-rpm-handlers-2.9.0-0.2.beta.el6.noarch.rpm | 73 kB 00:00
(12/26): pulp-rpm-plugins-2.9.0-0.2.beta.el6.noarch.rpm | 329 kB 00:00
(13/26): pulp-rpm-yumplugins-2.9.0-0.2.beta.el6.noarch.rpm | 45 kB 00:00
(14/26): pulp-selinux-2.9.0-0.2.beta.el6.noarch.rpm | 82 kB 00:00
(15/26): pulp-server-2.9.0-0.2.beta.el6.noarch.rpm | 833 kB 00:00
(16/26): python-kid-0.9.6-5.1.el6.noarch.rpm | 161 kB 00:00
(17/26): python-pulp-agent-lib-2.9.0-0.2.beta.el6.noarch.rpm | 91 kB 00:00
(18/26): python-pulp-bindings-2.9.0-0.2.beta.el6.noarch.rpm | 111 kB 00:00
(19/26): python-pulp-client-lib-2.9.0-0.2.beta.el6.noarch.rpm | 202 kB 00:00
(20/26): python-pulp-common-2.9.0-0.2.beta.el6.noarch.rpm | 123 kB 00:00
(21/26): python-pulp-oid_validation-2.9.0-0.2.beta.el6.noarch.rpm | 64 kB 00:00
(22/26): python-pulp-puppet-common-2.9.0-0.2.beta.el6.noarch.rpm | 32 kB 00:00
(23/26): python-pulp-repoauth-2.9.0-0.2.beta.el6.noarch.rpm | 77 kB 00:00
(24/26): python-pulp-rpm-common-2.9.0-0.2.beta.el6.noarch.rpm | 65 kB 00:00
(25/26): python-pulp-streamer-2.9.0-0.2.beta.el6.noarch.rpm | 74 kB 00:00
http://mirrors.nebo.edu/public/epel/6/x86_64/repoview-0.6.6-1.el6.noarch.rpm: [Errno 12] Timeout on http://mirrors.nebo.edu/public/epel/6/x86_64/repoview-0.6.6-1.el6.noarch.rpm: (28, 'connect() timed out!')
Trying other mirror.
(26/26): repoview-0.6.6-1.el6.noarch.rpm | 40 kB 00:00
---------------------------------------------------------------------------------------------------------
Total 93 kB/s | 3.0 MB 00:33
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : python-pulp-common-2.9.0-0.2.beta.el6.noarch 1/50
Updating : python-pulp-bindings-2.9.0-0.2.beta.el6.noarch 2/50
Updating : python-pulp-puppet-common-2.9.0-0.2.beta.el6.noarch 3/50
Updating : python-pulp-rpm-common-2.9.0-0.2.beta.el6.noarch 4/50
Updating : python-pulp-agent-lib-2.9.0-0.2.beta.el6.noarch 5/50
Updating : python-pulp-client-lib-2.9.0-0.2.beta.el6.noarch 6/50
Updating : pulp-consumer-client-2.9.0-0.2.beta.el6.noarch 7/50
Updating : pulp-admin-client-2.9.0-0.2.beta.el6.noarch 8/50
Updating : python-pulp-repoauth-2.9.0-0.2.beta.el6.noarch 9/50
Updating : python-pulp-oid_validation-2.9.0-0.2.beta.el6.noarch 10/50
Updating : pulp-selinux-2.9.0-0.2.beta.el6.noarch 11/50
Updating : pulp-server-2.9.0-0.2.beta.el6.noarch 12/50
Installing : python-kid-0.9.6-5.1.el6.noarch 13/50
Installing : repoview-0.6.6-1.el6.noarch 14/50
Updating : pulp-rpm-plugins-2.9.0-0.2.beta.el6.noarch 15/50
Updating : pulp-puppet-plugins-2.9.0-0.2.beta.el6.noarch 16/50
Updating : python-pulp-streamer-2.9.0-0.2.beta.el6.noarch 17/50
Updating : pulp-rpm-admin-extensions-2.9.0-0.2.beta.el6.noarch 18/50
Updating : pulp-puppet-admin-extensions-2.9.0-0.2.beta.el6.noarch 19/50
Updating : pulp-agent-2.9.0-0.2.beta.el6.noarch 20/50
Updating : pulp-puppet-consumer-extensions-2.9.0-0.2.beta.el6.noarch 21/50
Updating : pulp-rpm-consumer-extensions-2.9.0-0.2.beta.el6.noarch 22/50
Updating : pulp-puppet-handlers-2.9.0-0.2.beta.el6.noarch 23/50
Updating : pulp-rpm-handlers-2.9.0-0.2.beta.el6.noarch 24/50
Updating : pulp-rpm-yumplugins-2.9.0-0.2.beta.el6.noarch 25/50
Updating : grinder_test_package-5.0-1.fc14.noarch 26/50
Cleanup : pulp-puppet-plugins-2.8.5-0.1.beta.el6.noarch 27/50
Cleanup : pulp-puppet-admin-extensions-2.8.5-0.1.beta.el6.noarch 28/50
Cleanup : pulp-rpm-plugins-2.8.5-0.1.beta.el6.noarch 29/50
Cleanup : pulp-agent-2.8.5-0.1.beta.el6.noarch 30/50
Cleanup : pulp-puppet-consumer-extensions-2.8.5-0.1.beta.el6.noarch 31/50
Cleanup : pulp-rpm-handlers-2.8.5-0.1.beta.el6.noarch 32/50
Cleanup : pulp-rpm-admin-extensions-2.8.5-0.1.beta.el6.noarch 33/50
Cleanup : pulp-admin-client-2.8.5-0.1.beta.el6.noarch 34/50
Cleanup : pulp-puppet-handlers-2.8.5-0.1.beta.el6.noarch 35/50
Cleanup : python-pulp-agent-lib-2.8.5-0.1.beta.el6.noarch 36/50
Cleanup : python-pulp-rpm-common-2.8.5-0.1.beta.el6.noarch 37/50
Cleanup : python-pulp-puppet-common-2.8.5-0.1.beta.el6.noarch 38/50
Cleanup : python-pulp-oid_validation-2.8.5-0.1.beta.el6.noarch 39/50
Cleanup : pulp-rpm-consumer-extensions-2.8.5-0.1.beta.el6.noarch 40/50
Cleanup : pulp-consumer-client-2.8.5-0.1.beta.el6.noarch 41/50
Cleanup : python-pulp-client-lib-2.8.5-0.1.beta.el6.noarch 42/50
Cleanup : python-pulp-streamer-2.8.5-0.1.beta.el6.noarch 43/50
Cleanup : pulp-server-2.8.5-0.1.beta.el6.noarch 44/50
Cleanup : python-pulp-repoauth-2.8.5-0.1.beta.el6.noarch 45/50
Cleanup : pulp-rpm-yumplugins-2.8.5-0.1.beta.el6.noarch 46/50
Cleanup : python-pulp-bindings-2.8.5-0.1.beta.el6.noarch 47/50
Cleanup : python-pulp-common-2.8.5-0.1.beta.el6.noarch 48/50
Cleanup : pulp-selinux-2.8.5-0.1.beta.el6.noarch 49/50
Cleanup : grinder_test_package-4.0-1.fc14.noarch 50/50
Verifying : pulp-agent-2.9.0-0.2.beta.el6.noarch 1/50
Verifying : pulp-puppet-handlers-2.9.0-0.2.beta.el6.noarch 2/50
Verifying : python-kid-0.9.6-5.1.el6.noarch 3/50
Verifying : pulp-rpm-yumplugins-2.9.0-0.2.beta.el6.noarch 4/50
Verifying : pulp-rpm-plugins-2.9.0-0.2.beta.el6.noarch 5/50
Verifying : python-pulp-client-lib-2.9.0-0.2.beta.el6.noarch 6/50
Verifying : pulp-consumer-client-2.9.0-0.2.beta.el6.noarch 7/50
Verifying : python-pulp-repoauth-2.9.0-0.2.beta.el6.noarch 8/50
Verifying : python-pulp-bindings-2.9.0-0.2.beta.el6.noarch 9/50
Verifying : python-pulp-common-2.9.0-0.2.beta.el6.noarch 10/50
Verifying : grinder_test_package-5.0-1.fc14.noarch 11/50
Verifying : python-pulp-puppet-common-2.9.0-0.2.beta.el6.noarch 12/50
Verifying : pulp-rpm-admin-extensions-2.9.0-0.2.beta.el6.noarch 13/50
Verifying : pulp-rpm-handlers-2.9.0-0.2.beta.el6.noarch 14/50
Verifying : pulp-puppet-admin-extensions-2.9.0-0.2.beta.el6.noarch 15/50
Verifying : pulp-puppet-plugins-2.9.0-0.2.beta.el6.noarch 16/50
Verifying : pulp-selinux-2.9.0-0.2.beta.el6.noarch 17/50
Verifying : python-pulp-streamer-2.9.0-0.2.beta.el6.noarch 18/50
Verifying : pulp-puppet-consumer-extensions-2.9.0-0.2.beta.el6.noarch 19/50
Verifying : pulp-rpm-consumer-extensions-2.9.0-0.2.beta.el6.noarch 20/50
Verifying : python-pulp-oid_validation-2.9.0-0.2.beta.el6.noarch 21/50
Verifying : repoview-0.6.6-1.el6.noarch 22/50
Verifying : python-pulp-rpm-common-2.9.0-0.2.beta.el6.noarch 23/50
Verifying : python-pulp-agent-lib-2.9.0-0.2.beta.el6.noarch 24/50
Verifying : pulp-server-2.9.0-0.2.beta.el6.noarch 25/50
Verifying : pulp-admin-client-2.9.0-0.2.beta.el6.noarch 26/50
Verifying : pulp-rpm-handlers-2.8.5-0.1.beta.el6.noarch 27/50
Verifying : python-pulp-oid_validation-2.8.5-0.1.beta.el6.noarch 28/50
Verifying : python-pulp-bindings-2.8.5-0.1.beta.el6.noarch 29/50
Verifying : python-pulp-agent-lib-2.8.5-0.1.beta.el6.noarch 30/50
Verifying : pulp-consumer-client-2.8.5-0.1.beta.el6.noarch 31/50
Verifying : pulp-puppet-admin-extensions-2.8.5-0.1.beta.el6.noarch 32/50
Verifying : pulp-puppet-consumer-extensions-2.8.5-0.1.beta.el6.noarch 33/50
Verifying : python-pulp-rpm-common-2.8.5-0.1.beta.el6.noarch 34/50
Verifying : pulp-rpm-admin-extensions-2.8.5-0.1.beta.el6.noarch 35/50
Verifying : pulp-selinux-2.8.5-0.1.beta.el6.noarch 36/50
Verifying : pulp-rpm-plugins-2.8.5-0.1.beta.el6.noarch 37/50
Verifying : pulp-rpm-consumer-extensions-2.8.5-0.1.beta.el6.noarch 38/50
Verifying : python-pulp-puppet-common-2.8.5-0.1.beta.el6.noarch 39/50
Verifying : grinder_test_package-4.0-1.fc14.noarch 40/50
Verifying : pulp-rpm-yumplugins-2.8.5-0.1.beta.el6.noarch 41/50
Verifying : pulp-admin-client-2.8.5-0.1.beta.el6.noarch 42/50
Verifying : python-pulp-common-2.8.5-0.1.beta.el6.noarch 43/50
Verifying : python-pulp-repoauth-2.8.5-0.1.beta.el6.noarch 44/50
Verifying : pulp-agent-2.8.5-0.1.beta.el6.noarch 45/50
Verifying : python-pulp-streamer-2.8.5-0.1.beta.el6.noarch 46/50
Verifying : pulp-server-2.8.5-0.1.beta.el6.noarch 47/50
Verifying : pulp-puppet-plugins-2.8.5-0.1.beta.el6.noarch 48/50
Verifying : pulp-puppet-handlers-2.8.5-0.1.beta.el6.noarch 49/50
Verifying : python-pulp-client-lib-2.8.5-0.1.beta.el6.noarch 50/50
Dependency Installed:
python-kid.noarch 0:0.9.6-5.1.el6 repoview.noarch 0:0.6.6-1.el6
Updated:
grinder_test_package.noarch 0:5.0-1.fc14
pulp-admin-client.noarch 0:2.9.0-0.2.beta.el6
pulp-agent.noarch 0:2.9.0-0.2.beta.el6
pulp-consumer-client.noarch 0:2.9.0-0.2.beta.el6
pulp-puppet-admin-extensions.noarch 0:2.9.0-0.2.beta.el6
pulp-puppet-consumer-extensions.noarch 0:2.9.0-0.2.beta.el6
pulp-puppet-handlers.noarch 0:2.9.0-0.2.beta.el6
pulp-puppet-plugins.noarch 0:2.9.0-0.2.beta.el6
pulp-rpm-admin-extensions.noarch 0:2.9.0-0.2.beta.el6
pulp-rpm-consumer-extensions.noarch 0:2.9.0-0.2.beta.el6
pulp-rpm-handlers.noarch 0:2.9.0-0.2.beta.el6
pulp-rpm-plugins.noarch 0:2.9.0-0.2.beta.el6
pulp-rpm-yumplugins.noarch 0:2.9.0-0.2.beta.el6
pulp-selinux.noarch 0:2.9.0-0.2.beta.el6
pulp-server.noarch 0:2.9.0-0.2.beta.el6
python-pulp-agent-lib.noarch 0:2.9.0-0.2.beta.el6
python-pulp-bindings.noarch 0:2.9.0-0.2.beta.el6
python-pulp-client-lib.noarch 0:2.9.0-0.2.beta.el6
python-pulp-common.noarch 0:2.9.0-0.2.beta.el6
python-pulp-oid_validation.noarch 0:2.9.0-0.2.beta.el6
python-pulp-puppet-common.noarch 0:2.9.0-0.2.beta.el6
python-pulp-repoauth.noarch 0:2.9.0-0.2.beta.el6
python-pulp-rpm-common.noarch 0:2.9.0-0.2.beta.el6
python-pulp-streamer.noarch 0:2.9.0-0.2.beta.el6
Complete!
[root@~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.8 (Santiago)
[root@ ~]# rpm -qa |grep pulp-server
pulp-server-2.9.0-0.2.beta.el6.noarch
[root@ ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.8 (Santiago)
[root@ ~]# rpm -qa |grep pulp
python-pulp-python-common-1.1.2-0.2.beta.el6.noarch
pulp-python-admin-extensions-1.1.2-0.2.beta.el6.noarch
python-pulp-rpm-common-2.9.0-0.2.beta.el6.noarch
pulp-server-2.9.0-0.2.beta.el6.noarch
pulp-agent-2.9.0-0.2.beta.el6.noarch
python-kombu-3.0.33-5.pulp.el6.noarch
python-pulp-common-2.9.0-0.2.beta.el6.noarch
python-pulp-repoauth-2.9.0-0.2.beta.el6.noarch
python-pulp-streamer-2.9.0-0.2.beta.el6.noarch
pulp-rpm-yumplugins-2.9.0-0.2.beta.el6.noarch
mod_wsgi-3.4-2.pulp.el6.x86_64
python-pulp-bindings-2.9.0-0.2.beta.el6.noarch
python-pulp-client-lib-2.9.0-0.2.beta.el6.noarch
python-pulp-oid_validation-2.9.0-0.2.beta.el6.noarch
pulp-rpm-admin-extensions-2.9.0-0.2.beta.el6.noarch
pulp-rpm-consumer-extensions-2.9.0-0.2.beta.el6.noarch
python-isodate-0.5.0-4.pulp.el6.noarch
python-pulp-puppet-common-2.9.0-0.2.beta.el6.noarch
pulp-consumer-client-2.9.0-0.2.beta.el6.noarch
pulp-selinux-2.9.0-0.2.beta.el6.noarch
pulp-rpm-plugins-2.9.0-0.2.beta.el6.noarch
pulp-puppet-admin-extensions-2.9.0-0.2.beta.el6.noarch
pulp-puppet-handlers-2.9.0-0.2.beta.el6.noarch
python-pulp-docker-common-2.0.2-0.3.beta.el6.noarch
pulp-docker-plugins-2.0.2-0.3.beta.el6.noarch
pulp-docker-admin-extensions-2.0.2-0.3.beta.el6.noarch
pulp-python-plugins-1.1.2-0.2.beta.el6.noarch
pulp-admin-client-2.9.0-0.2.beta.el6.noarch
pulp-puppet-plugins-2.9.0-0.2.beta.el6.noarch
pulp-rpm-handlers-2.9.0-0.2.beta.el6.noarch
python-pulp-agent-lib-2.9.0-0.2.beta.el6.noarch
pulp-puppet-consumer-extensions-2.9.0-0.2.beta.el6.noarch
Updated by mhrivnak over 8 years ago
Looking at the machine where Preethi saw this, I was able to print out one of the XML snippets it was choking on:
<package arch="noarch" name="dog" pkgid="{{ pkgid }}">
<version epoch="0" rel="1" ver="4.23" />
</package
Note that the final ">" is missing.
I looked in the DB and saw that the record had been migrated, and both "repodata.other" and "repodata.filelists" were missing that last character.
> db.units_rpm.find({name: 'dog'}, {'repodata': 1})[0]
{
"_id" : "faaed075-f33e-44bc-bedf-7500e4f84e29",
"repodata" : {
"filelists" : "<package arch=\"noarch\" name=\"dog\" pkgid=\"{{ pkgid }}\">\n <version epoch=\"0\" rel=\"1\" ver=\"4.23\" />\n <file>/tmp/dog.txt</file>\n</package",
"other" : "<package arch=\"noarch\" name=\"dog\" pkgid=\"{{ pkgid }}\">\n <version epoch=\"0\" rel=\"1\" ver=\"4.23\" />\n</package",
"primary" : "<package type=\"rpm\">\n <name>dog</name>\n <arch>noarch</arch>\n <version epoch=\"0\" rel=\"1\" ver=\"4.23\" />\n <checksum pkgid=\"YES\" type=\"{{ checksumtype }}\">{{ checksum }}</checksum>\n <summary>A dummy package of dog</summary>\n <description>A dummy package of dog</description>\n <packager />\n <url>http://tstrachota.fedorapeople.org</url>\n <time build=\"1331831361\" file=\"1466751025\" />\n <size archive=\"292\" installed=\"42\" package=\"2423\" />\n <location href=\"dog-4.23-1.noarch.rpm\" />\n <format>\n <rpm:license xmlns:rpm=\"http://linux.duke.edu/metadata/rpm\">GPLv2</rpm:license>\n <rpm:vendor xmlns:rpm=\"http://linux.duke.edu/metadata/rpm\" />\n <rpm:group xmlns:rpm=\"http://linux.duke.edu/metadata/rpm\">Internet/Applications</rpm:group>\n <rpm:buildhost xmlns:rpm=\"http://linux.duke.edu/metadata/rpm\">smqe-ws15</rpm:buildhost>\n <rpm:sourcerpm xmlns:rpm=\"http://linux.duke.edu/metadata/rpm\">dog-4.23-1.src.rpm</rpm:sourcerpm>\n <rpm:header-range end=\"2277\" start=\"872\" xmlns:rpm=\"http://linux.duke.edu/metadata/rpm\" />\n <rpm:provides xmlns:rpm=\"http://linux.duke.edu/metadata/rpm\">\n <rpm:entry epoch=\"0\" flags=\"EQ\" name=\"dog\" rel=\"1\" ver=\"4.23\" />\n </rpm:provides>\n </format>\n</package>\n"
}
}
Updated by amacdona@redhat.com over 8 years ago
- Priority changed from High to Urgent
- Severity changed from 2. Medium to 3. High
- Triaged changed from No to Yes
Updated by mhrivnak over 8 years ago
I did reproduce this with an upgrade from 2.8.5 to 2.9.0 beta 2 and the same RHEL 5.11 repo linked to in the description.
Updated by mhrivnak over 8 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to mhrivnak
Updated by mhrivnak over 8 years ago
I hacked up the migration slightly to directly inject a repodata dict that was inducing the failure, and that's attached. You can see in the printed output that the generated "other" snippet is missing the last ">" character.
It behaves the same on both el6 and f22.
Updated by mhrivnak over 8 years ago
I have a fix that should be ready to go. I was able to trigger the bug via improvement to unit tests, and then see them pass with the fix. I just need to test the upgrade tomorrow.
Updated by mhrivnak over 8 years ago
For the curious, the reason this only breaks on very rare occasions comes down to white space. There is string slicing of XML going on, and the end point was found with a call to "index", which would return -1 if the string it was searching for wasn't found. In that case, the slice would just chop off the last character. This was happening for every single "other" and "filelists" snippet. For most of them, that was a white space character of some variety. But in rare occasions, it would be the final ">" character.
Added by mhrivnak over 8 years ago
Updated by mhrivnak over 8 years ago
- Status changed from ASSIGNED to POST
Updated by mhrivnak over 8 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset 3f4df31758ba60aea9f3a59c4f45de4d7aca6588.
Updated by pthomas@redhat.com over 8 years ago
verified
[root@dell-per320-01 ~]# sudo -u admin pulp-manage-db
sudo: unknown user: admin
sudo: unable to initialize policy plugin
[root@dell-per320-01 ~]# sudo -u apache pulp-manage-db
Attempting to connect to localhost:27017
Attempting to connect to localhost:27017
Write concern for Mongo connection: {}
/usr/lib/python2.6/site-packages/pulp/server/db/connection.py:159: DeprecationWarning: add_son_manipulator is deprecated
_DATABASE.add_son_manipulator(NamespaceInjector())
Loading content types.
Loading type descriptors []
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Loading unit model: docker_blob = pulp_docker.plugins.models:Blob
Loading unit model: docker_manifest = pulp_docker.plugins.models:Manifest
Loading unit model: docker_image = pulp_docker.plugins.models:Image
Loading unit model: docker_tag = pulp_docker.plugins.models:Tag
Loading unit model: python_package = pulp_python.plugins.models:Package
Loading unit model: puppet_module = pulp_puppet.plugins.db.models:Module
Loading unit model: erratum = pulp_rpm.plugins.db.models:Errata
Loading unit model: distribution = pulp_rpm.plugins.db.models:Distribution
Loading unit model: srpm = pulp_rpm.plugins.db.models:SRPM
Loading unit model: package_group = pulp_rpm.plugins.db.models:PackageGroup
Loading unit model: package_category = pulp_rpm.plugins.db.models:PackageCategory
Loading unit model: iso = pulp_rpm.plugins.db.models:ISO
Loading unit model: package_environment = pulp_rpm.plugins.db.models:PackageEnvironment
Loading unit model: drpm = pulp_rpm.plugins.db.models:DRPM
Loading unit model: package_langpacks = pulp_rpm.plugins.db.models:PackageLangpacks
Loading unit model: rpm = pulp_rpm.plugins.db.models:RPM
Loading unit model: yum_repo_metadata_file = pulp_rpm.plugins.db.models:YumMetadataFile
Updating the database with types []
/usr/lib/python2.6/site-packages/pulp/server/db/model/base.py:96: DeprecationWarning: ensure_index is deprecated. Use create_index instead.
unique=unique, background=True)
Found the following type definitions that were not present in the update collection [puppet_module, drpm, docker_manifest, docker_blob, erratum, yum_repo_metadata_file, package_group, package_category, iso, package_environment, docker_tag, python_package, srpm, rpm, distribution, docker_image]
Updating the database with types [puppet_module, drpm, package_langpacks, docker_manifest, docker_blob, erratum, yum_repo_metadata_file, package_group, package_category, iso, package_environment, docker_tag, python_package, distribution, rpm, srpm, docker_image]
/usr/lib/python2.6/site-packages/pulp/plugins/types/database.py:277: DeprecationWarning: save is deprecated. Use insert_one or replace_one instead
content_type_collection.save(content_type)
Content types loaded.
Ensuring the admin role and user are in place.
Admin role and user are in place.
Beginning database migrations.
Applying pulp.server.db.migrations version 24
/usr/lib/python2.6/site-packages/pulp/server/db/migrations/0024_distributor_schema_change.py:23: DeprecationWarning: save is deprecated. Use insert_one or replace_one instead
collection.save(distributor)
/usr/lib/python2.6/site-packages/mongoengine/document.py:367: DeprecationWarning: update is deprecated. Use replace_one, update_one or update_many instead.
upsert=upsert, **write_concern)
Migration to pulp.server.db.migrations version 24 complete.
Migration package pulp_docker.plugins.migrations is up to date at version 2
Migration package pulp_puppet.plugins.migrations is up to date at version 5
Migration package pulp_python.plugins.migrations is up to date at version 1
Applying pulp_rpm.plugins.migrations version 33
Migration to pulp_rpm.plugins.migrations version 33 complete.
Loading unit model: docker_blob = pulp_docker.plugins.models:Blob
Loading unit model: docker_manifest = pulp_docker.plugins.models:Manifest
Loading unit model: docker_image = pulp_docker.plugins.models:Image
Loading unit model: docker_tag = pulp_docker.plugins.models:Tag
Loading unit model: python_package = pulp_python.plugins.models:Package
Loading unit model: puppet_module = pulp_puppet.plugins.db.models:Module
Loading unit model: erratum = pulp_rpm.plugins.db.models:Errata
Loading unit model: distribution = pulp_rpm.plugins.db.models:Distribution
Loading unit model: srpm = pulp_rpm.plugins.db.models:SRPM
Loading unit model: package_group = pulp_rpm.plugins.db.models:PackageGroup
Loading unit model: package_category = pulp_rpm.plugins.db.models:PackageCategory
Loading unit model: iso = pulp_rpm.plugins.db.models:ISO
Loading unit model: package_environment = pulp_rpm.plugins.db.models:PackageEnvironment
Loading unit model: drpm = pulp_rpm.plugins.db.models:DRPM
Loading unit model: package_langpacks = pulp_rpm.plugins.db.models:PackageLangpacks
Loading unit model: rpm = pulp_rpm.plugins.db.models:RPM
Loading unit model: yum_repo_metadata_file = pulp_
<//pre>
Updated by pthomas@redhat.com over 8 years ago
- Status changed from MODIFIED to 6
Updated by semyers over 8 years ago
- Status changed from 6 to CLOSED - CURRENTRELEASE
"other" and "filelists" XML snippet processing fixed to not do slicing
These snippets were mistakenly being sent through the "remove_fake_element" function, which would slice them incorrectly and shave off a single character. This was ok most of the time, because it would be a white space character. But occasionally it was the final ">" of the XML.
fixes #2037 https://pulp.plan.io/issues/2037