Project

Profile

Help

Issue #2754

RPM uploads appear to be missing metadata information

Added by daviddavis over 4 years ago. Updated almost 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
2.14.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 23
Quarter:

Description

Specifically, the fields Size, Summary, Source RPM, and Group appear to be missing.

Steps:

$ wget ftp://195.220.108.108/linux/fedora-secondary/development/rawhide/Everything/aarch64/os/Packages/v/vim-enhanced-8.0.596-1.fc27.aarch64.rpm
$ rpm -qip vim-enhanced-8.0.596-1.fc27.aarch64.rpm
...
Group       : Applications/Editors
Size        : 3028853
...
Source RPM  : vim-8.0.596-1.fc27.src.rpm
...
Summary     : A version of the VIM editor which includes recent enhancements
...
$ pulp-admin rpm repo uploads rpm --file vim-enhanced-8.0.596-1.fc27.aarch64.rpm --repo-id zoo
$ phttp https://localhost/pulp/api/v2/repositories/zoo/search/units/ criteria:='{"fields": {"unit": ["name", "summary", "size", "group", "sourcerpm"]}, "type_ids": ["rpm"], "filters": {"unit": {"name"
: "vim-enhanced"}}}'                                                                                                                                                                                             
[
    {
        "_id": {
            "$oid": "5915aac507425414f84ec276"
        },
        "created": "2017-05-12T12:29:57Z",
        "metadata": {
            "_content_type_id": "rpm",
            "_id": "939240d9-2071-451d-8749-c10cac057f31",
            "name": "vim-enhanced"
        },
        "repo_id": "zoo",
        "unit_id": "939240d9-2071-451d-8749-c10cac057f31",
        "unit_type_id": "rpm",
        "updated": "2017-05-12T12:29:57Z"
    }
]

I checked the database too and the fields aren't there.

Since this is also a Katello issue, we should notify them if they need to be setting particular metadata fields on their import upload request too.

Associated revisions

Revision 62db898a View on GitHub
Added by fdobrovo over 4 years ago

Fix extracting metadata for rpm/srpm upload

fixes #2754 https://pulp.plan.io/issues/2754

History

#2 Updated by daviddavis over 4 years ago

  • Description updated (diff)

Just for comparison, here's another repo that was synced (not uploaded);

$ phttp https://localhost/pulp/api/v2/repositories/zoo/search/units/ criteria:='{"fields": {"unit": ["name", "summary", "size", "group", "sourcerpm"]}, "type_ids": ["rpm"], "filters": {"unit": {"name": "kangaroo"}}}'                                                                                                                                                                                                

[
    {
        "_id": {
            "$oid": "5913618307425414f84ec20f"
        },
        "created": "2017-05-10T18:52:51Z",
        "metadata": {
            "_content_type_id": "rpm",
            "_id": "460666b7-6c66-4e8e-80a1-da0af2374ebd",
            "group": "Internet/Applications",
            "name": "kangaroo",
            "size": 2467,
            "sourcerpm": "kangaroo-0.2-1.src.rpm",
            "summary": "A dummy package of kangaroo"
        },
        "repo_id": "zoo",
        "unit_id": "460666b7-6c66-4e8e-80a1-da0af2374ebd",
        "unit_type_id": "rpm",
        "updated": "2017-05-10T18:52:51Z"
    }
]

#3 Updated by daviddavis over 4 years ago

  • Description updated (diff)

#4 Updated by amacdona@redhat.com over 4 years ago

  • Triaged changed from No to Yes

#5 Updated by mhrivnak over 4 years ago

This was a similar issue. No idea if they're related: https://pulp.plan.io/issues/2365

#6 Updated by daviddavis over 4 years ago

I cleaned my database and tried uploading the kangaroo package from comment 2:

wget https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/kangaroo-0.2-1.noarch.rpm
pulp-admin rpm repo uploads rpm --file kangaroo-0.2-1.noarch.rpm --repo-id zoo
phttp https://localhost/pulp/api/v2/repositories/zoo/search/units/ criteria:='{"fields": {"unit": ["name", "summary", "size", "group", "sourcerpm"]}, "type_ids": ["rpm"], "filters": {"unit": {"name": "kangaroo"}}}' 

Result:

[
    {
        "_id": {
            "$oid": "5915d12607425414f84ec27c"
        },
        "created": "2017-05-12T15:13:42Z",
        "metadata": {
            "_content_type_id": "rpm",
            "_id": "230fd66e-abca-429f-b651-e7ff9c59ab96",
            "name": "kangaroo"
        },
        "repo_id": "zoo",
        "unit_id": "230fd66e-abca-429f-b651-e7ff9c59ab96",
        "unit_type_id": "rpm",
        "updated": "2017-05-12T15:13:42Z"
    }
]

#7 Updated by daviddavis over 4 years ago

Digging into the code a bit, it looks like we parse the headers for uploaded content in yum upload importer[0]. These headers seem to contain the information we need:

In [16]: headers['group']
Out[16]: 'Internet/Applications'

In [17]: headers['summary']
Out[17]: 'A dummy package of dolphin'

In [18]: headers['name']
Out[18]: 'dolphin'

In [19]: headers['size']
Out[19]: 42L

In [20]: headers['sourcerpm']
Out[20]: 'dolphin-3.10.232-1.src.rpm'

However, it appears looking at say the CLI, Pulp doesn't care about these values[1].

Given that a user reported this though, I think we should maybe try to (at the very least) store these values in the database and return them in the API for uploaded rpms (we already do this for synced rpms). I am not sure what sort of implications that has though.

[0] https://github.com/pulp/pulp_rpm/blob/master/plugins/pulp_rpm/plugins/importers/yum/upload.py#L482
[1] https://github.com/pulp/pulp_rpm/blob/master/extensions_admin/pulp_rpm/extensions/admin/contents.py#L31-L33

#8 Updated by mhrivnak over 4 years ago

  • Sprint/Milestone set to 38

#9 Updated by fdobrovo over 4 years ago

  • Status changed from NEW to ASSIGNED

#10 Updated by fdobrovo over 4 years ago

  • Assignee set to fdobrovo

#11 Updated by mhrivnak over 4 years ago

  • Sprint/Milestone changed from 38 to 39

#12 Updated by ttereshc over 4 years ago

  • Status changed from ASSIGNED to NEW
  • Assignee deleted (fdobrovo)

#14 Updated by fdobrovo over 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fdobrovo

#15 Updated by mhrivnak over 4 years ago

  • Sprint/Milestone changed from 39 to 40

#16 Updated by mhrivnak over 4 years ago

  • Sprint/Milestone changed from 40 to 41

#17 Updated by fdobrovo over 4 years ago

  • Status changed from ASSIGNED to POST

#18 Updated by mhrivnak over 4 years ago

  • Sprint/Milestone changed from 41 to 42

#19 Updated by fdobrovo over 4 years ago

  • Status changed from POST to MODIFIED

#20 Updated by pcreech over 4 years ago

  • Platform Release set to 2.14.0

#21 Updated by pcreech over 4 years ago

  • Status changed from MODIFIED to 5

#22 Updated by pthomas@redhat.com over 4 years ago

The automated test for this issue is passing.

#23 Updated by pcreech over 4 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

#24 Updated by bmbouter almost 4 years ago

  • Sprint set to Sprint 23

#25 Updated by bmbouter almost 4 years ago

  • Sprint/Milestone deleted (42)

#26 Updated by bmbouter almost 3 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF