Issue #1505
closedfields missing when pulp-admin tries to present a Distribution
Description
pulp-admin is expecting each of the files in a distribution to have a 'size' and 'filename' field[0], but those fields aren't present in the API response. This is reproducible using the same testing steps from #1478:
Create a repo with distribution units:
pulp-admin rpm repo create --repo-id rawhide --feed http://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/x86_64/os/ --skip=rpm,drpm,erratum
pulp-admin rpm repo sync run --repo-id rawhide
Now try to list the distros:
pulp-admin -vv rpm repo content distribution --repo-id rawhide
You should see a traceback:
Traceback (most recent call last):
File "/home/vagrant/devel/pulp/client_lib/pulp/client/extensions/core.py", line 474, in run
exit_code = Cli.run(self, args)
File "/usr/lib/python2.7/site-packages/okaara/cli.py", line 974, in run
exit_code = command_or_section.execute(self.prompt, remaining_args)
File "/home/vagrant/devel/pulp/client_lib/pulp/client/extensions/extensions.py", line 210, in execute
return self.method(*arg_list, **clean_kwargs)
File "/home/vagrant/devel/pulp_rpm/extensions_admin/pulp_rpm/extensions/admin/contents.py", line 337, in distribution
self.run_search([TYPE_DISTRIBUTION], self.write_distro, **kwargs)
File "/home/vagrant/devel/pulp_rpm/extensions_admin/pulp_rpm/extensions/admin/contents.py", line 171, in run_search
out_func(units)
File "/home/vagrant/devel/pulp_rpm/extensions_admin/pulp_rpm/extensions/admin/contents.py", line 375, in write_distro
'filename': f['filename'],
KeyError: 'filename'
Here's the object pulp-admin is trying to process:
{ "files": [
{
"relativepath": "images/pxeboot/initrd.img",
"checksumtype": "sha256",
"checksum": "3ade414b6ea0640cbd1caf1fb41d28f44b62b8813269c10e89c6b9ff51b1fc46"
},
{
"relativepath": "images/pxeboot/vmlinuz",
"checksumtype": "sha256",
"checksum": "fa7b19fd885c3adf66f18e65e89c12111f248feff95b62310a1b6fcdff45e2cd"
},
{
"relativepath": "images/boot.iso",
"checksumtype": "sha256",
"checksum": "f5e752772cb4d2e19c0cd0c825314524bcf2651bcdc8df8a6a6f03dbeb5fbc7e"
},
{
"relativepath": "images/efiboot.img",
"checksumtype": "sha256",
"checksum": "7f1fe2754d34045c9627e562b83bfd16c5c4d11213b34b186905375499e176fa"
},
{
"relativepath": "images/macboot.img",
"checksumtype": "sha256",
"checksum": "c90e3612c5519fb779c82995e66f1375e560e1af385e01c6002438f5fe9ac7df"
},
{
"relativepath": "images/install.img",
"checksumtype": "sha256",
"checksum": "520d8ce0175427f9fd536c805b15e5e710a6052793b66724b32d01ad807defa2"
}
],
"_storage_path": "/var/lib/pulp/content/units/distribution/82d9/82d90e56-a944-40e2-bd09-04baac7fafcd",
"family": "Fedora",
"timestamp": 1452249124.2275844,
"_last_updated": 1452283534,
"packagedir": "",
"variant": "",
"downloaded": true,
"version": "rawhide",
"version_sort_index": "$rawhide",
"pulp_user_metadata": {},
"_content_type_id": "distribution",
"_ns": "units_distribution",
"_id": "82d90e56-a944-40e2-bd09-04baac7fafcd",
"arch": "x86_64",
"id": "ks-Fedora--rawhide-x86_64"
}
Distributions appear to support adding these fields to the files property, but the method that does the work doesn't appear to have been called in the example above.
Note: This issue is masked by the problem that #1478 addresses, so make sure that that issue is MODIFIED or the changeset referenced by that issue is applied to your test env before testing.
[0]: https://github.com/pulp/pulp_rpm/blob/27f045257b883b8c39d8f31c4fd5660c9990d1e7/extensions_admin/pulp_rpm/extensions/admin/contents.py#L373-L380
[1]: https://github.com/pulp/pulp_rpm/blob/27f045257b883b8c39d8f31c4fd5660c9990d1e7/plugins/pulp_rpm/plugins/db/models.py#L251-L278
Related issues
Updated by semyers almost 9 years ago
- Related to Issue #1478: POST request to /pulp/api/v2/repositories/<reponame>/search/units/ does not transform id correctly when unit results returned added
Added by semyers almost 9 years ago
Added by semyers almost 9 years ago
Revision c7e79237 | View on GitHub
RepoUnitSearch remaps unit fields using legacy field remapper
The reproducer in #1478 will still fail, but with a problem that appears to be specific to the distribution type in pulp_rpm. That problem is being tracked in https://pulp.plan.io/issues/1505
https://pulp.plan.io/issues/1478 fixes #1478
re #1505
Updated by semyers almost 9 years ago
- Version set to Master
- Platform Release set to 2.8.0
Updated by semyers almost 9 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to semyers
/me rolls up sleeves and dives in
Updated by semyers almost 9 years ago
Adding these fields to the distribution files list was removed in the lazy conversion[0]. They previously came as properties of the download report when non-lazily syncing the repo. To support laziness, it seems like pulp-admin needs to allow for the possibility that distribution files might not have a known filename or size.
[0]: https://github.com/pulp/pulp_rpm/commit/7715f91c7488a0d34b74174fd5e4e485b0251e2b
Updated by semyers almost 9 years ago
Alright. I've checked with our buds over at katello, and they pretty confident that these fields going missing doesn't hurt their workflow, but can most likely deal with it if it's a breaking change. When I asked the team about their removal, jortel said "I removed those values because best I could tell they were not used (and some were wrong)".
Given all of that, I'm just going to remove them from pulp-admin.
Added by semyers almost 9 years ago
Revision 62a78aeb | View on GitHub
pulp-admin updated to reflect removed distribution fields
These setting were removed when pulp_rpm implemented lazy downloading, but pulp-admin was still trying to use them.
Updated by semyers almost 9 years ago
- Status changed from ASSIGNED to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulp_rpm:62a78aeb641b6edce5099db989be57ffba9dc82b.
Updated by dkliban@redhat.com almost 9 years ago
- Status changed from MODIFIED to 5
Updated by dkliban@redhat.com almost 9 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
RepoUnitSearch remaps unit fields using legacy field remapper
The reproducer in #1478 will still fail, but with a problem that appears to be specific to the distribution type in pulp_rpm. That problem is being tracked in https://pulp.plan.io/issues/1505
https://pulp.plan.io/issues/1478 fixes #1478
re #1505