Issue #2330: `KeyError: 'url'` is raised when syncing. Pulp silently ignores the error. - Python Support - Pulp

Actions

Send by e-mail Copy link

Issue #2330

closed

`KeyError: 'url'` is raised when syncing. Pulp silently ignores the error.

Added by Ichimonji10 almost 7 years ago. Updated over 4 years ago.

Status:

CLOSED - NOTABUG

Priority:

Normal

Assignee:

Sprint/Milestone:

Start date:

Due date:

Estimated time:

Severity:

2. Medium

Version - Python:

master

Platform Release:

Target Release - Python:

OS:

Triaged:

Groomed:

Sprint Candidate:

Tags:

Pulp 2

Sprint:

Quarter:

Description

Try executing the following:

pulp-admin python repo create --repo-id foo --feed https://pypi.python.org/ --package-names shelf-reader
pulp-admin python repo sync run --repo-id foo
pulp-admin python repo packages --repo-id foo
pulp-admin python repo delete --repo-id foo

It'll work. Now, do the same with a feed URL of https://repos.fedorapeople.org/pulp/pulp/fixtures/python/, and it'll fail. Pulp will give no indication that anything has wrong, but the sync will pull in no packages. journalctl indicates why the sync doesn't pull in packages:

Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120) 'url'
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120) Traceback (most recent call last):
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120)   File "/usr/lib/python2.7/site-packages/nectar/downloaders/base.py", line 145, in _fire_event_to_listener
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120)     event_listener_callback(*args, **kwargs)
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120)   File "/usr/lib/python2.7/site-packages/pulp_python/plugins/importers/sync.py", line 68, in download_succeeded
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120)     self._process_manifest(destination.read())
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120)   File "/usr/lib/python2.7/site-packages/pulp_python/plugins/importers/sync.py", line 108, in _process_manifest
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120)     url = package['url']
Oct 11 10:49:58 pulp.example.com pulp[4838]: nectar.downloaders.base:ERROR: (4838-41120) KeyError: 'url'

Pulp should be able to sync without raising a KeyError. In my opinion, Pulp shouldn't silently ignore the error.

Actions

Copy link

Updated by amacdona@redhat.com almost 7 years ago

Status changed from NEW to CLOSED - NOTABUG

The code path in the traceback is in the 1.1-dev branch, but is not in current master. The pulp_python master branch will correspond to pulp_python 2.0.

The python data in pulp_fixtures is generated by a pulp publish. Unfortunately, pulp to pulp sync is introduced in pulp_python 2.0 so this data cannot work for earlier code.

This has revealed other work that needs to be done but I don't think this is a bug. I have filed the following issues:

pulp smash: skip and rename python sync test for pulp_python < 2.0 [0]
pulp python: ensure version is incremented everywhere it needs to be.
pulp fixtures: Create a second python fixture that mirrors pypi data rather than a pulp publish.
pulp smash: Add/modify pulp smash tests so they sync with pypi data

[0] https://github.com/PulpQE/pulp-smash/issues/416
[1] https://pulp.plan.io/issues/2333
[2] https://github.com/PulpQE/pulp-fixtures/issues/40
[3] https://github.com/PulpQE/pulp-smash/issues/417

Actions

Copy link