Project

Profile

Help

Issue #1505

fields missing when pulp-admin tries to present a Distribution

Added by semyers over 4 years ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Master
Platform Release:
2.8.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

pulp-admin is expecting each of the files in a distribution to have a 'size' and 'filename' field[0], but those fields aren't present in the API response. This is reproducible using the same testing steps from #1478:

Create a repo with distribution units:

pulp-admin rpm repo create --repo-id rawhide --feed http://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/x86_64/os/ --skip=rpm,drpm,erratum
pulp-admin rpm repo sync run --repo-id rawhide

Now try to list the distros:

pulp-admin -vv rpm repo content distribution --repo-id rawhide

You should see a traceback:

Traceback (most recent call last):
  File "/home/vagrant/devel/pulp/client_lib/pulp/client/extensions/core.py", line 474, in run
    exit_code = Cli.run(self, args)
  File "/usr/lib/python2.7/site-packages/okaara/cli.py", line 974, in run
    exit_code = command_or_section.execute(self.prompt, remaining_args)
  File "/home/vagrant/devel/pulp/client_lib/pulp/client/extensions/extensions.py", line 210, in execute
    return self.method(*arg_list, **clean_kwargs)
  File "/home/vagrant/devel/pulp_rpm/extensions_admin/pulp_rpm/extensions/admin/contents.py", line 337, in distribution
    self.run_search([TYPE_DISTRIBUTION], self.write_distro, **kwargs)
  File "/home/vagrant/devel/pulp_rpm/extensions_admin/pulp_rpm/extensions/admin/contents.py", line 171, in run_search
    out_func(units)
  File "/home/vagrant/devel/pulp_rpm/extensions_admin/pulp_rpm/extensions/admin/contents.py", line 375, in write_distro
    'filename': f['filename'],
KeyError: 'filename'

Here's the object pulp-admin is trying to process:

    { "files": [
        {
          "relativepath": "images/pxeboot/initrd.img", 
          "checksumtype": "sha256", 
          "checksum": "3ade414b6ea0640cbd1caf1fb41d28f44b62b8813269c10e89c6b9ff51b1fc46"
        }, 
        {
          "relativepath": "images/pxeboot/vmlinuz", 
          "checksumtype": "sha256", 
          "checksum": "fa7b19fd885c3adf66f18e65e89c12111f248feff95b62310a1b6fcdff45e2cd"
        }, 
        {
          "relativepath": "images/boot.iso", 
          "checksumtype": "sha256", 
          "checksum": "f5e752772cb4d2e19c0cd0c825314524bcf2651bcdc8df8a6a6f03dbeb5fbc7e"
        }, 
        {
          "relativepath": "images/efiboot.img", 
          "checksumtype": "sha256", 
          "checksum": "7f1fe2754d34045c9627e562b83bfd16c5c4d11213b34b186905375499e176fa"
        }, 
        {
          "relativepath": "images/macboot.img", 
          "checksumtype": "sha256", 
          "checksum": "c90e3612c5519fb779c82995e66f1375e560e1af385e01c6002438f5fe9ac7df"
        }, 
        {
          "relativepath": "images/install.img", 
          "checksumtype": "sha256", 
          "checksum": "520d8ce0175427f9fd536c805b15e5e710a6052793b66724b32d01ad807defa2"
        }
      ], 
      "_storage_path": "/var/lib/pulp/content/units/distribution/82d9/82d90e56-a944-40e2-bd09-04baac7fafcd", 
      "family": "Fedora", 
      "timestamp": 1452249124.2275844, 
      "_last_updated": 1452283534, 
      "packagedir": "", 
      "variant": "", 
      "downloaded": true, 
      "version": "rawhide", 
      "version_sort_index": "$rawhide", 
      "pulp_user_metadata": {}, 
      "_content_type_id": "distribution", 
      "_ns": "units_distribution", 
      "_id": "82d90e56-a944-40e2-bd09-04baac7fafcd", 
      "arch": "x86_64", 
      "id": "ks-Fedora--rawhide-x86_64"
    }

Distributions appear to support adding these fields to the files property, but the method that does the work doesn't appear to have been called in the example above.

Note: This issue is masked by the problem that #1478 addresses, so make sure that that issue is MODIFIED or the changeset referenced by that issue is applied to your test env before testing.

[0]: https://github.com/pulp/pulp_rpm/blob/27f045257b883b8c39d8f31c4fd5660c9990d1e7/extensions_admin/pulp_rpm/extensions/admin/contents.py#L373-L380
[1]: https://github.com/pulp/pulp_rpm/blob/27f045257b883b8c39d8f31c4fd5660c9990d1e7/plugins/pulp_rpm/plugins/db/models.py#L251-L278


Related issues

Related to RPM Support - Issue #1478: POST request to /pulp/api/v2/repositories/<reponame>/search/units/ does not transform id correctly when unit results returnedCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision c7e79237 View on GitHub
Added by semyers over 4 years ago

RepoUnitSearch remaps unit fields using legacy field remapper

The reproducer in #1478 will still fail, but with a problem that appears to be specific to the distribution type in pulp_rpm. That problem is being tracked in https://pulp.plan.io/issues/1505

https://pulp.plan.io/issues/1478 fixes #1478

re #1505

Revision c7e79237 View on GitHub
Added by semyers over 4 years ago

RepoUnitSearch remaps unit fields using legacy field remapper

The reproducer in #1478 will still fail, but with a problem that appears to be specific to the distribution type in pulp_rpm. That problem is being tracked in https://pulp.plan.io/issues/1505

https://pulp.plan.io/issues/1478 fixes #1478

re #1505

Revision 62a78aeb View on GitHub
Added by semyers over 4 years ago

pulp-admin updated to reflect removed distribution fields

These setting were removed when pulp_rpm implemented lazy downloading, but pulp-admin was still trying to use them.

https://pulp.plan.io/issues/1505 fixes #1505

History

#1 Updated by semyers over 4 years ago

  • Related to Issue #1478: POST request to /pulp/api/v2/repositories/<reponame>/search/units/ does not transform id correctly when unit results returned added

#2 Updated by semyers over 4 years ago

  • Version set to Master
  • Platform Release set to 2.8.0

#3 Updated by semyers over 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to semyers

/me rolls up sleeves and dives in

#4 Updated by semyers over 4 years ago

Adding these fields to the distribution files list was removed in the lazy conversion[0]. They previously came as properties of the download report when non-lazily syncing the repo. To support laziness, it seems like pulp-admin needs to allow for the possibility that distribution files might not have a known filename or size.

[0]: https://github.com/pulp/pulp_rpm/commit/7715f91c7488a0d34b74174fd5e4e485b0251e2b

#5 Updated by semyers over 4 years ago

Alright. I've checked with our buds over at katello, and they pretty confident that these fields going missing doesn't hurt their workflow, but can most likely deal with it if it's a breaking change. When I asked the team about their removal, jortel said "I removed those values because best I could tell they were not used (and some were wrong)".

Given all of that, I'm just going to remove them from pulp-admin.

#6 Updated by semyers over 4 years ago

  • Status changed from ASSIGNED to MODIFIED
  • % Done changed from 0 to 100

#7 Updated by mhrivnak over 4 years ago

  • Triaged changed from No to Yes

#8 Updated by dkliban@redhat.com over 4 years ago

  • Status changed from MODIFIED to 5

#9 Updated by dkliban@redhat.com over 4 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

#10 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF