Project

Profile

Help

Issue #2330

`KeyError: 'url'` is raised when syncing. Pulp silently ignores the error.

Added by Ichimonji10 almost 4 years ago. Updated over 1 year ago.

Status:
CLOSED - NOTABUG
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version - Python:
master
Platform Release:
Target Release - Python:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Try executing the following:

pulp-admin python repo create --repo-id foo --feed https://pypi.python.org/ --package-names shelf-reader
pulp-admin python repo sync run --repo-id foo
pulp-admin python repo packages --repo-id foo
pulp-admin python repo delete --repo-id foo

It'll work. Now, do the same with a feed URL of https://repos.fedorapeople.org/pulp/pulp/fixtures/python/, and it'll fail. Pulp will give no indication that anything has wrong, but the sync will pull in no packages. journalctl indicates why the sync doesn't pull in packages:

Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120) 'url'
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120) Traceback (most recent call last):
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120)   File "/usr/lib/python2.7/site-packages/nectar/downloaders/base.py", line 145, in _fire_event_to_listener
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120)     event_listener_callback(*args, **kwargs)
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120)   File "/usr/lib/python2.7/site-packages/pulp_python/plugins/importers/sync.py", line 68, in download_succeeded
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120)     self._process_manifest(destination.read())
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120)   File "/usr/lib/python2.7/site-packages/pulp_python/plugins/importers/sync.py", line 108, in _process_manifest
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120)     url = package['url']
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120) KeyError: 'url'

Pulp should be able to sync without raising a KeyError. In my opinion, Pulp shouldn't silently ignore the error.

History

#1 Updated by amacdona@redhat.com almost 4 years ago

  • Status changed from NEW to CLOSED - NOTABUG

The code path in the traceback is in the 1.1-dev branch, but is not in current master. The pulp_python master branch will correspond to pulp_python 2.0.

The python data in pulp_fixtures is generated by a pulp publish. Unfortunately, pulp to pulp sync is introduced in pulp_python 2.0 so this data cannot work for earlier code.

This has revealed other work that needs to be done but I don't think this is a bug. I have filed the following issues:

  • pulp smash: skip and rename python sync test for pulp_python < 2.0 [0]
  • pulp python: ensure version is incremented everywhere it needs to be.
  • pulp fixtures: Create a second python fixture that mirrors pypi data rather than a pulp publish.
  • pulp smash: Add/modify pulp smash tests so they sync with pypi data

[0] https://github.com/PulpQE/pulp-smash/issues/416
[1] https://pulp.plan.io/issues/2333
[2] https://github.com/PulpQE/pulp-fixtures/issues/40
[3] https://github.com/PulpQE/pulp-smash/issues/417

#2 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF