RPM Support

A user reported on the foreman forum https://community.theforeman.org/t/failure-to-sync-fedora-32-repos-pulp-yum-updater-error-command-document-too-large/20023

I’m attempting to sync Fedora 32 repositories from the Princeton mirror (I have tried another just to rule out the repo.) Unfortunately the sync fails with the exception Katello::Errors::PulpError: PLP0000: Importer indicated a failed response and the Error tab shows that the Pulp yum_importer failed with error command document too large.

I’ve found a handful of similar sounding bug reports, but they’re from a few years ago and claim to have fixed the issue. https://access.redhat.com/solutions/2957371 1

I have two reproduced this on two systems, Katello 3.15.3 and RH Satellite 6.7.2.

Expected outcome: Repository to sync correctly as it did with Fedora 32.

Reproducible?: If it’s not something with me, then you should just have to add the Fedora 32 Everything repo to a Product and attempt to sync. It’ll process metadata for a while and then error out.

Foreman and Proxy versions: Foreman 2.0.1 w/ Katello 3.15.3 also produced the error on Satellite 6.7.2

Distribution and version: Katello = CentOS 7.8 Satellite = RHEL 7.8

Other relevant data:

{“pulp_tasks”=>
[{“exception”=>nil,
“task_type”=>“pulp.server.managers.repo.sync.sync”,
“_href”=>"/pulp/api/v2/tasks/91caf2e6-6d0d-4dfb-abcd-d0e3879e40bb/",
“task_id”=>“91caf2e6-6d0d-4dfb-abcd-d0e3879e40bb”,
“tags”=>
[“pulp:repository:156f4383-2ae0-48cb-96ec-462eee4c4c22”,
“pulp:action:sync”],
“finish_time”=>“2020-08-10T20:39:33Z”,
“ns"=>“task_status”,
“start_time”=>“2020-08-10T20:23:43Z”,
“traceback”=>
“Traceback (most recent call last):\n” +
" File “/usr/lib/python2.7/site-packages/celery/app/trace.py”, line 367, in trace_task\n" +
" R = retval = fun(*args, **kwargs)\n" +
" File “/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py”, line 686, in call\n" +
" return super(Task, self).call(*args, **kwargs)\n" +
" File “/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py”, line 108, in call\n" +
" return super(PulpTask, self).call(*args, **kwargs)\n" +
" File “/usr/lib/python2.7/site-packages/celery/app/trace.py”, line 622, in protected_call\n" +
" return self.run(*args, **kwargs)\n" +
" File “/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py”, line 855, in sync\n" +
" raise pulp_exceptions.PulpExecutionException((‘Importer indicated a failed response’))\n” +
“PulpExecutionException: Importer indicated a failed response\n”,
“spawned_tasks”=>,
“progress_report”=>
{“yum_importer”=>
{“content”=>
{“size_total”=>78681195936,
“items_left”=>11641,
“items_total”=>55327,
“state”=>“FAILED”,
“size_left”=>15786547076,
“details”=>
{“rpm_total”=>55327,
“rpm_done”=>43686,
“drpm_total”=>0,
“drpm_done”=>0},
“error”=>“command document too large”,
“error_details”=>},
“comps”=>{“state”=>“NOT_STARTED”},
“purge_duplicates”=>{“state”=>“NOT_STARTED”},
“distribution”=>
{“items_total”=>0,
“state”=>“NOT_STARTED”,
“error_details”=>,
“items_left”=>0},
“modules”=>{“state”=>“NOT_STARTED”},
“errata”=>{“state”=>“NOT_STARTED”},
“metadata”=>{“state”=>“FINISHED”}}},
“queue”=>“reserved_resource_worker-1@libkatello.library.unt.edu.dq2”,
“state”=>“error”,
“worker_name”=>“reserved_resource_worker-1@libkatello.library.unt.edu”,
“result”=>nil,
“error”=>
{“code”=>“PLP0000”,
“data”=>{},
“description”=>“Importer indicated a failed response”,
“sub_errors”=>},
“_id”=>{"$oid"=>“5f31accf9aae893f3610f6fd”},
“id”=>“5f31accf9aae893f3610f6fd”}],
“contents_changed”=>true,
“poll_attempts”=>{“total”=>80, “failed”=>1}}

Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064) command document too large
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064) Traceback (most recent call last):
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 294, in run
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)     self.update_content(metadata_files, url)
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 607, in update_content
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)     self.download_rpms(metadata_files, rpms_to_download, url)
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 834, in download_rpms
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)     unit = self.add_rpm_unit(metadata_files, unit)
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 759, in add_rpm_unit
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)     unit.save()
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)   File "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 324, in save
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)     object_id = collection.save(doc, **write_concern)
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)   File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 2180, in save
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)     check_keys, False, manipulate, write_concern)
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)   File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 709, in _update
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)     codec_options=self.codec_options).copy()
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)   File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 216, in command
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)     self._raise_connection_failure(error)
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)   File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 343, in _raise_connection_failure
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064)     raise error
Aug 11 15:20:04 host.fqdn pulp[6556]: pulp_rpm.plugins.importers.yum.sync:ERROR: [7181f260] (6556-80064) DocumentTooLarge: command document too large