Project

Profile

Help

Issue #2234

closed

Upgrading to pulp 2.9 causes sync of repos with SRPM's with a filelist field to fail.

Added by akegata@gmail.com over 6 years ago. Updated over 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Urgent
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
2.9.2
Platform Release:
2.9.3
OS:
RHEL 6
Triaged:
Yes
Groomed:
Yes
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 8
Quarter:

Description

After upgrading to 2.9.2, some of our repos cannot sync due to a field which should probably not be there on SRPM:

Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) The field 'filelist' does not exist on the document 'SRPM'
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) Traceback (most recent call last):
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 222, in run
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)     self.update_content(metadata_files, url)
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 485, in update_content
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)     rpms_to_download, drpms_to_download = self._decide_what_to_download(metadata_files, catalog)
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 508, in _decide_what_to_download
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)     self._decide_rpms_to_download(metadata_files, catalog)
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 546, in _decide_rpms_to_download
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)     wanted.iterkeys(), self.conduit, self.download_deferred, catalog)
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/existing.py", line 120, in check_all_and_associate
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)     for unit in units_controller.find_units(unit_generator):
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)   File "/usr/lib/python2.6/site-packages/pulp/server/controllers/units.py", line 40, in find_units
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)     for found_unit in query:
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)   File "/usr/lib/python2.6/site-packages/mongoengine/queryset/base.py", line 1411, in next
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)     _auto_dereference=self._auto_dereference, only_fields=self.only_fields)
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)   File "/usr/lib/python2.6/site-packages/mongoengine/base/document.py", line 730, in _from_son
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)     obj = cls(__auto_convert=False, _created=created, __only_fields=only_fields, **data)
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/db/models.py", line 731, in __init__
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)     super(RpmBase, self).__init__(*args, **kwargs)
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/db/models.py", line 149, in __init__
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)     super(NonMetadataPackage, self).__init__(*args, **kwargs)
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)   File "/usr/lib/python2.6/site-packages/pulp/server/db/model/__init__.py", line 55, in __init__
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)     super(AutoRetryDocument, self).__init__(*args, **kwargs)
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)   File "/usr/lib/python2.6/site-packages/mongoengine/base/document.py", line 80, in __init__
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656)     raise FieldDoesNotExist(msg)
Sep  7 14:38:58 vinf0039 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (13980-26656) FieldDoesNotExist: The field 'filelist' does not exist on the document 'SRPM'
Sep  7 14:38:59 vinf0039 pulp: pulp.server.event.http:INFO: (13980-26656) {"call_report": {"exception": null, "task_type": "pulp.server.managers.repo.sync.sync", "_href": "/pulp/api/v2/tasks/f0c50fb8-55b1-49a9-a81e-0dcff3a1f850/", "task_id": "f0c50fb8-55b1-49a9-a81e-0dcff3a1f850", "tags": ["pulp:repository:OL6_Latest_x86_64_temp", "pulp:action:sync"], "finish_time": null, "_ns": "task_status", "start_time": "2016-09-07T12:18:32Z", "traceback": null, "spawned_tasks": [], "progress_report": {"yum_importer": {"content": {"size_total": 0, "items_left": 0, "items_total": 0, "state": "FAILED", "size_left": 0, "details": {"rpm_total": 0, "rpm_done": 0, "drpm_total": 0, "drpm_done": 0}, "error": "The field 'filelist' does not exist on the document 'SRPM'", "error_details": []}, "comps": {"state": "NOT_STARTED"}, "purge_duplicates": {"state": "NOT_STARTED"}, "distribution": {"items_total": 0, "state": "NOT_STARTED", "error_details": [], "items_left": 0}, "errata": {"state": "NOT_STARTED"}, "metadata": {"state": "FINISHED"}}}, "state": "running", "worker_name": "reserved_resource_worker-5@server", "result": null, "error": null, "_id": {"$oid": "57d00598b858e1aded1f9736"}, "id": "57d00598b858e1aded1f9736"}, "event_type": "repo.sync.finish", "payload": {"importer_id": "yum_importer", "exception": null, "repo_id": "OL6_Latest_x86_64_temp", "traceback": null, "started": "2016-09-07T12:18:32Z", "_ns": "repo_sync_results", "completed": "2016-09-07T12:38:59Z", "importer_type_id": "yum_importer", "error_message": null, "summary": {"content": {"state": "FAILED"}, "comps": {"state": "NOT_STARTED"}, "purge_duplicates": {"state": "NOT_STARTED"}, "distribution": {"state": "NOT_STARTED"}, "errata": {"state": "NOT_STARTED"}, "metadata": {"state": "FINISHED"}}, "added_count": 720, "result": "failed", "updated_count": 0, "details": {"content": {"size_total": 0, "items_left": 0, "items_total": 0, "state": "FAILED", "size_left": 0, "details": {"rpm_total": 0, "rpm_done": 0, "drpm_total": 0, "drpm_done": 0}, "error": "The field 'filelist' doe
Sep  7 14:38:59 vinf0039 pulp: pulp.server.event.http:INFO: (13980-26656) s not exist on the document 'SRPM'", "error_details": []}, "comps": {"state": "NOT_STARTED"}, "purge_duplicates": {"state": "NOT_STARTED"}, "distribution": {"items_total": 0, "state": "NOT_STARTED", "error_details": [], "items_left": 0}, "errata": {"state": "NOT_STARTED"}, "metadata": {"state": "FINISHED"}}, "id": "57d00a63357058369ceab967", "removed_count": 0}}

Related issues

Related to RPM Support - Issue #2236: Upgrading to pulp 2.9 causes sync of repos with DRPM' with a "relativepath" field to fail.CLOSED - CURRENTRELEASEipanova@redhat.comActions
Actions #1

Updated by bmbouter over 6 years ago

We had this problem for the RPM collection as issue #1952. The solution is to write a migration similar to this one[0] except that it needs to be for units_srpm instead of units_rpm.

[0]: https://github.com/pulp/pulp_rpm/commit/a520375868e84128a965099dd5fbd8a383c807ac#diff-3f47f9dcad7d4c06b6e701e7f1330729R9

Actions #2

Updated by bmbouter over 6 years ago

As a workaround for this issue you could connect to mongod and switch to using the pulp_database and run the following command. This is effectively what the migration will do. It will be save to run this manually now and if a migration is made it can also run later. Consider taking a backup before running this.

db.units_srpm.update({'filelist': {'$exists': true}}, {'$unset': {'filelist': true}}, multi=true)
Actions #3

Updated by ipanova@redhat.com over 6 years ago

Would be good to check if drpm collection also needs to a migration to be written.

Actions #4

Updated by ipanova@redhat.com over 6 years ago

  • Related to Issue #2236: Upgrading to pulp 2.9 causes sync of repos with DRPM' with a "relativepath" field to fail. added
Actions #5

Updated by amacdona@redhat.com over 6 years ago

  • Priority changed from Normal to Urgent
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes
Actions #6

Updated by ttereshc over 6 years ago

  • Sprint/Milestone set to 26
Actions #7

Updated by ipanova@redhat.com about 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ipanova@redhat.com
Actions #8

Updated by akegata@gmail.com about 6 years ago

bmbouter wrote:

As a workaround for this issue you could connect to mongod and switch to using the pulp_database and run the following command. This is effectively what the migration will do. It will be save to run this manually now and if a migration is made it can also run later. Consider taking a backup before running this.

I tried this workaround, but it did not work. The filelist field is still present in the database for the repo in question.

Added by ipanova@redhat.com about 6 years ago

Revision b8213d55

Remove no longer used filelist filed from srpm/drpm collections.

closes #2234 https://pulp.plan.io/issues/2234

Actions #9

Updated by ipanova@redhat.com about 6 years ago

wrote:

bmbouter wrote:

As a workaround for this issue you could connect to mongod and switch to using the pulp_database and run the following command. This is effectively what the migration will do. It will be save to run this manually now and if a migration is made it can also run later. Consider taking a backup before running this.

I tried this workaround, but it did not work. The filelist field is still present in the database for the repo in question.

You should run this command not just against SRPM but DRPM also.

Actions #10

Updated by ipanova@redhat.com about 6 years ago

  • Status changed from ASSIGNED to POST
Actions #11

Updated by akegata@gmail.com about 6 years ago

wrote:

You should run this command not just against SRPM but DRPM also.

That makes sense, but it doesn't seem to match anything in my case:

db.units_srpm.update({'filelist': {'$exists': true}}, {'$unset': {'filelist': true}}, multi=true)

WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

db.units_drpm.update({'filelist': {'$exists': true}}, {'$unset': {'filelist': true}}, multi=true)

WriteResult({
"nMatched" : 0,
"nUpserted" : 1,
"nModified" : 0,
"_id" : ObjectId("57d7b337b71f5981b8328a1c")
})

Just for good measure I tried to sync the repository I'm having issues with again, it still fails.

Actions #12

Updated by ipanova@redhat.com about 6 years ago

I don't know how that is possible, can you paste the traceback from the sync? if it is different or the same.

Actions #13

Updated by ipanova@redhat.com about 6 years ago

i see where is the issue, use updateMany, from your query i see that it modified just one unit.

Actions #14

Updated by ipanova@redhat.com about 6 years ago

  • Status changed from POST to MODIFIED
Actions #15

Updated by semyers about 6 years ago

  • Platform Release set to 2.9.3
Actions #16

Updated by akegata@gmail.com about 6 years ago

wrote:

i see where is the issue, use updateMany, from your query i see that it modified just one unit.

This worked. Running this updated several entries and the sync continued past this stage:

db.units_srpm.updateMany({'filelist': {'$exists': true}}, {'$unset': {'filelist': true}}, multi=true)

Added by ipanova@redhat.com about 6 years ago

Revision 9496b438

Remove no longer used filelist filed from srpm/drpm collections.

closes #2234 https://pulp.plan.io/issues/2234

Actions #18

Updated by semyers about 6 years ago

  • Status changed from MODIFIED to 5
Actions #19

Updated by pthomas@redhat.com about 6 years ago

  • Status changed from 5 to 6

verified
Upgrade job has been updated to cover this case.

Actions #20

Updated by semyers about 6 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE
Actions #22

Updated by bmbouter almost 5 years ago

  • Sprint set to Sprint 8
Actions #23

Updated by bmbouter almost 5 years ago

  • Sprint/Milestone deleted (26)
Actions #24

Updated by bmbouter over 3 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF