Project

Profile

Help

Issue #1505

closed

fields missing when pulp-admin tries to present a Distribution

Added by semyers about 8 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Master
Platform Release:
2.8.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

pulp-admin is expecting each of the files in a distribution to have a 'size' and 'filename' field[0], but those fields aren't present in the API response. This is reproducible using the same testing steps from #1478:

Create a repo with distribution units:

pulp-admin rpm repo create --repo-id rawhide --feed http://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/x86_64/os/ --skip=rpm,drpm,erratum
pulp-admin rpm repo sync run --repo-id rawhide

Now try to list the distros:

pulp-admin -vv rpm repo content distribution --repo-id rawhide

You should see a traceback:

Traceback (most recent call last):
  File "/home/vagrant/devel/pulp/client_lib/pulp/client/extensions/core.py", line 474, in run
    exit_code = Cli.run(self, args)
  File "/usr/lib/python2.7/site-packages/okaara/cli.py", line 974, in run
    exit_code = command_or_section.execute(self.prompt, remaining_args)
  File "/home/vagrant/devel/pulp/client_lib/pulp/client/extensions/extensions.py", line 210, in execute
    return self.method(*arg_list, **clean_kwargs)
  File "/home/vagrant/devel/pulp_rpm/extensions_admin/pulp_rpm/extensions/admin/contents.py", line 337, in distribution
    self.run_search([TYPE_DISTRIBUTION], self.write_distro, **kwargs)
  File "/home/vagrant/devel/pulp_rpm/extensions_admin/pulp_rpm/extensions/admin/contents.py", line 171, in run_search
    out_func(units)
  File "/home/vagrant/devel/pulp_rpm/extensions_admin/pulp_rpm/extensions/admin/contents.py", line 375, in write_distro
    'filename': f['filename'],
KeyError: 'filename'

Here's the object pulp-admin is trying to process:

    { "files": [
        {
          "relativepath": "images/pxeboot/initrd.img", 
          "checksumtype": "sha256", 
          "checksum": "3ade414b6ea0640cbd1caf1fb41d28f44b62b8813269c10e89c6b9ff51b1fc46"
        }, 
        {
          "relativepath": "images/pxeboot/vmlinuz", 
          "checksumtype": "sha256", 
          "checksum": "fa7b19fd885c3adf66f18e65e89c12111f248feff95b62310a1b6fcdff45e2cd"
        }, 
        {
          "relativepath": "images/boot.iso", 
          "checksumtype": "sha256", 
          "checksum": "f5e752772cb4d2e19c0cd0c825314524bcf2651bcdc8df8a6a6f03dbeb5fbc7e"
        }, 
        {
          "relativepath": "images/efiboot.img", 
          "checksumtype": "sha256", 
          "checksum": "7f1fe2754d34045c9627e562b83bfd16c5c4d11213b34b186905375499e176fa"
        }, 
        {
          "relativepath": "images/macboot.img", 
          "checksumtype": "sha256", 
          "checksum": "c90e3612c5519fb779c82995e66f1375e560e1af385e01c6002438f5fe9ac7df"
        }, 
        {
          "relativepath": "images/install.img", 
          "checksumtype": "sha256", 
          "checksum": "520d8ce0175427f9fd536c805b15e5e710a6052793b66724b32d01ad807defa2"
        }
      ], 
      "_storage_path": "/var/lib/pulp/content/units/distribution/82d9/82d90e56-a944-40e2-bd09-04baac7fafcd", 
      "family": "Fedora", 
      "timestamp": 1452249124.2275844, 
      "_last_updated": 1452283534, 
      "packagedir": "", 
      "variant": "", 
      "downloaded": true, 
      "version": "rawhide", 
      "version_sort_index": "$rawhide", 
      "pulp_user_metadata": {}, 
      "_content_type_id": "distribution", 
      "_ns": "units_distribution", 
      "_id": "82d90e56-a944-40e2-bd09-04baac7fafcd", 
      "arch": "x86_64", 
      "id": "ks-Fedora--rawhide-x86_64"
    }

Distributions appear to support adding these fields to the files property, but the method that does the work doesn't appear to have been called in the example above.

Note: This issue is masked by the problem that #1478 addresses, so make sure that that issue is MODIFIED or the changeset referenced by that issue is applied to your test env before testing.

[0]: https://github.com/pulp/pulp_rpm/blob/27f045257b883b8c39d8f31c4fd5660c9990d1e7/extensions_admin/pulp_rpm/extensions/admin/contents.py#L373-L380
[1]: https://github.com/pulp/pulp_rpm/blob/27f045257b883b8c39d8f31c4fd5660c9990d1e7/plugins/pulp_rpm/plugins/db/models.py#L251-L278


Related issues

Related to RPM Support - Issue #1478: POST request to /pulp/api/v2/repositories/<reponame>/search/units/ does not transform id correctly when unit results returnedCLOSED - CURRENTRELEASEsemyersActions
Actions #1

Updated by semyers about 8 years ago

  • Related to Issue #1478: POST request to /pulp/api/v2/repositories/<reponame>/search/units/ does not transform id correctly when unit results returned added

Added by semyers about 8 years ago

Revision c7e79237 | View on GitHub

RepoUnitSearch remaps unit fields using legacy field remapper

The reproducer in #1478 will still fail, but with a problem that appears to be specific to the distribution type in pulp_rpm. That problem is being tracked in https://pulp.plan.io/issues/1505

https://pulp.plan.io/issues/1478 fixes #1478

re #1505

Added by semyers about 8 years ago

Revision c7e79237 | View on GitHub

RepoUnitSearch remaps unit fields using legacy field remapper

The reproducer in #1478 will still fail, but with a problem that appears to be specific to the distribution type in pulp_rpm. That problem is being tracked in https://pulp.plan.io/issues/1505

https://pulp.plan.io/issues/1478 fixes #1478

re #1505

Actions #2

Updated by semyers about 8 years ago

  • Version set to Master
  • Platform Release set to 2.8.0
Actions #3

Updated by semyers about 8 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to semyers

/me rolls up sleeves and dives in

Actions #4

Updated by semyers about 8 years ago

Adding these fields to the distribution files list was removed in the lazy conversion[0]. They previously came as properties of the download report when non-lazily syncing the repo. To support laziness, it seems like pulp-admin needs to allow for the possibility that distribution files might not have a known filename or size.

[0]: https://github.com/pulp/pulp_rpm/commit/7715f91c7488a0d34b74174fd5e4e485b0251e2b

Actions #5

Updated by semyers about 8 years ago

Alright. I've checked with our buds over at katello, and they pretty confident that these fields going missing doesn't hurt their workflow, but can most likely deal with it if it's a breaking change. When I asked the team about their removal, jortel said "I removed those values because best I could tell they were not used (and some were wrong)".

Given all of that, I'm just going to remove them from pulp-admin.

Added by semyers about 8 years ago

Revision 62a78aeb | View on GitHub

pulp-admin updated to reflect removed distribution fields

These setting were removed when pulp_rpm implemented lazy downloading, but pulp-admin was still trying to use them.

https://pulp.plan.io/issues/1505 fixes #1505

Actions #6

Updated by semyers about 8 years ago

  • Status changed from ASSIGNED to MODIFIED
  • % Done changed from 0 to 100
Actions #7

Updated by mhrivnak about 8 years ago

  • Triaged changed from No to Yes
Actions #8

Updated by dkliban@redhat.com about 8 years ago

  • Status changed from MODIFIED to 5
Actions #9

Updated by dkliban@redhat.com about 8 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #10

Updated by bmbouter almost 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF