Project

Profile

Help

Issue #4262

closed

Pulp raises an unhelpful IOError when it fails to download a repo metadata file

Added by daviddavis over 5 years ago. Updated about 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
2.19.0
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

We're trying to debug a sync error downstream but it's impossible since Pulp raises an unhelpful IOError. See this stacktrace:

Nov 27 17:13:30 eggnog1234 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://satellite.example.com/pulp/repos/OrgA/STAGING/CV_CSL3_1_1/content/dist/rhel/server/6/6Server/x86_64/rhscl/1/os/repodata/productid failed with code 404: Not Found
Nov 27 17:13:31 eggnog1234 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040) Not Found
Nov 27 17:13:31 eggnog1234 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040) Traceback (most recent call last):
Nov 27 17:13:31 eggnog1234 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 263, in run
Nov 27 17:13:31 eggnog1234 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040)     metadata_files = self.get_metadata(metadata_files)
Nov 27 17:13:31 eggnog1234 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 450, in get_metadata
Nov 27 17:13:31 eggnog1234 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040)     metadata_files.download_metadata_files()
Nov 27 17:13:31 eggnog1234 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/metadata.py", line 217, in download_metadata_files
Nov 27 17:13:31 eggnog1234 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040)     raise IOError(error_report.error_msg)
Nov 27 17:13:31 eggnog1234 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040) IOError: Not Found
Nov 27 17:13:31 eggnog1234 pulp: pulp.server.async.tasks:INFO: [acf7b76c] Task failed : [acf7b76c-61ca-4950-acd4-a6da29500c90]
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040) Task pulp.server.managers.repo.sync.sync[acf7b76c-61ca-4950-acd4-a6da29500c90] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040) Traceback (most recent call last):
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040)     R = retval = fun(*args, **kwargs)
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 527, in __call__
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040)     return super(Task, self).__call__(*args, **kwargs)
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040)     return super(PulpTask, self).__call__(*args, **kwargs)
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040)     return self.run(*args, **kwargs)
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 827, in sync
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040)     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Nov 27 17:13:31 eggnog1234 pulp: celery.worker.job:ERROR: (23088-27040) PulpExecutionException: Importer indicated a failed response

I'm not sure if the failed download of productid is related to the IOError. Also, IOError seems like the wrong exception to be raised here[0].

[0] https://github.com/pulp/pulp_rpm/blob/9bc9e734a9c19dcd9865546a3a66645acb9a8d01/plugins/pulp_rpm/plugins/importers/yum/repomd/metadata.py#L224


Related issues

Related to RPM Support - Test #4579: Automate rpm missing filelistCLOSED - COMPLETErochacbrunoActions
Actions #2

Updated by daviddavis over 5 years ago

Steps to reproduce:

pulp-admin rpm repo create --repo-id missing-filelists --feed https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-missing-filelists/
pulp-admin rpm repo sync run --repo-id missing-filelists
Actions #3

Updated by daviddavis over 5 years ago

  • Status changed from NEW to POST
  • Assignee set to daviddavis

Added by daviddavis over 5 years ago

Revision 9deecb60 | View on GitHub

Improve the error message for missing metadata files

fixes #4262 https://pulp.plan.io/issues/4262

Actions #4

Updated by daviddavis over 5 years ago

  • Status changed from POST to MODIFIED
Actions #5

Updated by ttereshc about 5 years ago

  • Platform Release set to 2.19.0
Actions #6

Updated by ttereshc about 5 years ago

  • Sprint/Milestone set to 2.19.0
Actions #7

Updated by ttereshc about 5 years ago

  • Status changed from MODIFIED to 5
Actions #8

Updated by rochacbruno about 5 years ago

  • Related to Test #4579: Automate rpm missing filelist added
Actions #9

Updated by ttereshc about 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #10

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF