Issue #2234
closed
Upgrading to pulp 2.9 causes sync of repos with SRPM's with a filelist field to fail.
Description
After upgrading to 2.9.2, some of our repos cannot sync due to a field which should probably not be there on SRPM:
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) The field 'filelist' does not exist on the document 'SRPM'
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) Traceback (most recent call last):
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 222, in run
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) self.update_content(metadata_files, url)
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 485, in update_content
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) rpms_to_download, drpms_to_download = self._decide_what_to_download(metadata_files, catalog)
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 508, in _decide_what_to_download
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) self._decide_rpms_to_download(metadata_files, catalog)
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 546, in _decide_rpms_to_download
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) wanted.iterkeys(), self.conduit, self.download_deferred, catalog)
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/existing.py", line 120, in check_all_and_associate
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) for unit in units_controller.find_units(unit_generator):
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) File "/usr/lib/python2.6/site-packages/pulp/server/controllers/units.py", line 40, in find_units
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) for found_unit in query:
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) File "/usr/lib/python2.6/site-packages/mongoengine/queryset/base.py", line 1411, in next
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) _auto_dereference=self._auto_dereference, only_fields=self.only_fields)
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) File "/usr/lib/python2.6/site-packages/mongoengine/base/document.py", line 730, in _from_son
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) obj = cls(__auto_convert=False, _created=created, __only_fields=only_fields, **data)
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/db/models.py", line 731, in __init__
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) super(RpmBase, self).__init__(*args, **kwargs)
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/db/models.py", line 149, in __init__
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) super(NonMetadataPackage, self).__init__(*args, **kwargs)
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) File "/usr/lib/python2.6/site-packages/pulp/server/db/model/__init__.py", line 55, in __init__
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) super(AutoRetryDocument, self).__init__(*args, **kwargs)
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) File "/usr/lib/python2.6/site-packages/mongoengine/base/document.py", line 80, in __init__
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) raise FieldDoesNotExist(msg)
Sep 7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) FieldDoesNotExist: The field 'filelist' does not exist on the document 'SRPM'
Sep 7 14:38:59 vinf0039 pulp: pulp.server.event.http:INFO: (13980-26656) {"call_report": {"exception": null, "task_type": "pulp.server.managers.repo.sync.sync", "_href": "/pulp/api/v2/tasks/f0c50fb8-55b1-49a9-a81e-0dcff3a1f850/", "task_id": "f0c50fb8-55b1-49a9-a81e-0dcff3a1f850", "tags": ["pulp:repository:OL6_Latest_x86_64_temp", "pulp:action:sync"], "finish_time": null, "_ns": "task_status", "start_time": "2016-09-07T12:18:32Z", "traceback": null, "spawned_tasks": [], "progress_report": {"yum_importer": {"content": {"size_total": 0, "items_left": 0, "items_total": 0, "state": "FAILED", "size_left": 0, "details": {"rpm_total": 0, "rpm_done": 0, "drpm_total": 0, "drpm_done": 0}, "error": "The field 'filelist' does not exist on the document 'SRPM'", "error_details": []}, "comps": {"state": "NOT_STARTED"}, "purge_duplicates": {"state": "NOT_STARTED"}, "distribution": {"items_total": 0, "state": "NOT_STARTED", "error_details": [], "items_left": 0}, "errata": {"state": "NOT_STARTED"}, "metadata": {"state": "FINISHED"}}}, "state": "running", "worker_name": "reserved_resource_worker-5@server", "result": null, "error": null, "_id": {"$oid": "57d00598b858e1aded1f9736"}, "id": "57d00598b858e1aded1f9736"}, "event_type": "repo.sync.finish", "payload": {"importer_id": "yum_importer", "exception": null, "repo_id": "OL6_Latest_x86_64_temp", "traceback": null, "started": "2016-09-07T12:18:32Z", "_ns": "repo_sync_results", "completed": "2016-09-07T12:38:59Z", "importer_type_id": "yum_importer", "error_message": null, "summary": {"content": {"state": "FAILED"}, "comps": {"state": "NOT_STARTED"}, "purge_duplicates": {"state": "NOT_STARTED"}, "distribution": {"state": "NOT_STARTED"}, "errata": {"state": "NOT_STARTED"}, "metadata": {"state": "FINISHED"}}, "added_count": 720, "result": "failed", "updated_count": 0, "details": {"content": {"size_total": 0, "items_left": 0, "items_total": 0, "state": "FAILED", "size_left": 0, "details": {"rpm_total": 0, "rpm_done": 0, "drpm_total": 0, "drpm_done": 0}, "error": "The field 'filelist' doe
Sep 7 14:38:59 vinf0039 pulp: pulp.server.event.http:INFO: (13980-26656) s not exist on the document 'SRPM'", "error_details": []}, "comps": {"state": "NOT_STARTED"}, "purge_duplicates": {"state": "NOT_STARTED"}, "distribution": {"items_total": 0, "state": "NOT_STARTED", "error_details": [], "items_left": 0}, "errata": {"state": "NOT_STARTED"}, "metadata": {"state": "FINISHED"}}, "id": "57d00a63357058369ceab967", "removed_count": 0}}
Related issues
Updated by bmbouter over 8 years ago
We had this problem for the RPM collection as issue #1952. The solution is to write a migration similar to this one[0] except that it needs to be for units_srpm instead of units_rpm.
Updated by bmbouter over 8 years ago
As a workaround for this issue you could connect to mongod and switch to using the pulp_database and run the following command. This is effectively what the migration will do. It will be save to run this manually now and if a migration is made it can also run later. Consider taking a backup before running this.
db.units_srpm.update({'filelist': {'$exists': true}}, {'$unset': {'filelist': true}}, multi=true)
Updated by ipanova@redhat.com over 8 years ago
Would be good to check if drpm collection also needs to a migration to be written.
Updated by ipanova@redhat.com over 8 years ago
- Related to Issue #2236: Upgrading to pulp 2.9 causes sync of repos with DRPM' with a "relativepath" field to fail. added
Updated by amacdona@redhat.com over 8 years ago
- Priority changed from Normal to Urgent
- Severity changed from 2. Medium to 3. High
- Triaged changed from No to Yes
Updated by ipanova@redhat.com over 8 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to ipanova@redhat.com
Updated by akegata@gmail.com over 8 years ago
bmbouter wrote:
As a workaround for this issue you could connect to mongod and switch to using the pulp_database and run the following command. This is effectively what the migration will do. It will be save to run this manually now and if a migration is made it can also run later. Consider taking a backup before running this.
I tried this workaround, but it did not work. The filelist field is still present in the database for the repo in question.
Added by ipanova@redhat.com over 8 years ago
Updated by ipanova@redhat.com over 8 years ago
akegata@gmail.com wrote:
bmbouter wrote:
As a workaround for this issue you could connect to mongod and switch to using the pulp_database and run the following command. This is effectively what the migration will do. It will be save to run this manually now and if a migration is made it can also run later. Consider taking a backup before running this.
I tried this workaround, but it did not work. The filelist field is still present in the database for the repo in question.
You should run this command not just against SRPM but DRPM also.
Updated by ipanova@redhat.com over 8 years ago
- Status changed from ASSIGNED to POST
Updated by akegata@gmail.com over 8 years ago
ipanova@redhat.com wrote:
You should run this command not just against SRPM but DRPM also.
That makes sense, but it doesn't seem to match anything in my case:
db.units_srpm.update({'filelist': {'$exists': true}}, {'$unset': {'filelist': true}}, multi=true)
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
db.units_drpm.update({'filelist': {'$exists': true}}, {'$unset': {'filelist': true}}, multi=true)
WriteResult({
"nMatched" : 0,
"nUpserted" : 1,
"nModified" : 0,
"_id" : ObjectId("57d7b337b71f5981b8328a1c")
})
Just for good measure I tried to sync the repository I'm having issues with again, it still fails.
Updated by ipanova@redhat.com over 8 years ago
I don't know how that is possible, can you paste the traceback from the sync? if it is different or the same.
Updated by ipanova@redhat.com over 8 years ago
i see where is the issue, use updateMany, from your query i see that it modified just one unit.
Updated by ipanova@redhat.com over 8 years ago
- Status changed from POST to MODIFIED
Applied in changeset b8213d55c302df6c6e2babac439e4e89de699345.
Updated by akegata@gmail.com over 8 years ago
ipanova@redhat.com wrote:
i see where is the issue, use updateMany, from your query i see that it modified just one unit.
This worked. Running this updated several entries and the sync continued past this stage:
db.units_srpm.updateMany({'filelist': {'$exists': true}}, {'$unset': {'filelist': true}}, multi=true)
Added by ipanova@redhat.com over 8 years ago
Revision 9496b438 | View on GitHub
Remove no longer used filelist filed from srpm/drpm collections.
Updated by ipanova@redhat.com over 8 years ago
Applied in changeset 9496b4384c6c67d3b27f65597a67887cb4236164.
Updated by pthomas@redhat.com over 8 years ago
- Status changed from 5 to 6
verified
Upgrade job has been updated to cover this case.
Updated by semyers over 8 years ago
- Status changed from 6 to CLOSED - CURRENTRELEASE
.
Remove no longer used filelist filed from srpm/drpm collections.
closes #2234 https://pulp.plan.io/issues/2234