Project

Profile

Help

Issue #2754

closed

RPM uploads appear to be missing metadata information

Added by daviddavis over 5 years ago. Updated almost 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
2.14.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 23
Quarter:

Description

Specifically, the fields Size, Summary, Source RPM, and Group appear to be missing.

Steps:

$ wget ftp://195.220.108.108/linux/fedora-secondary/development/rawhide/Everything/aarch64/os/Packages/v/vim-enhanced-8.0.596-1.fc27.aarch64.rpm
$ rpm -qip vim-enhanced-8.0.596-1.fc27.aarch64.rpm
...
Group       : Applications/Editors
Size        : 3028853
...
Source RPM  : vim-8.0.596-1.fc27.src.rpm
...
Summary     : A version of the VIM editor which includes recent enhancements
...
$ pulp-admin rpm repo uploads rpm --file vim-enhanced-8.0.596-1.fc27.aarch64.rpm --repo-id zoo
$ phttp https://localhost/pulp/api/v2/repositories/zoo/search/units/ criteria:='{"fields": {"unit": ["name", "summary", "size", "group", "sourcerpm"]}, "type_ids": ["rpm"], "filters": {"unit": {"name"
: "vim-enhanced"}}}'                                                                                                                                                                                             
[
    {
        "_id": {
            "$oid": "5915aac507425414f84ec276"
        },
        "created": "2017-05-12T12:29:57Z",
        "metadata": {
            "_content_type_id": "rpm",
            "_id": "939240d9-2071-451d-8749-c10cac057f31",
            "name": "vim-enhanced"
        },
        "repo_id": "zoo",
        "unit_id": "939240d9-2071-451d-8749-c10cac057f31",
        "unit_type_id": "rpm",
        "updated": "2017-05-12T12:29:57Z"
    }
]

I checked the database too and the fields aren't there.

Since this is also a Katello issue, we should notify them if they need to be setting particular metadata fields on their import upload request too.

Actions #2

Updated by daviddavis over 5 years ago

  • Description updated (diff)

Just for comparison, here's another repo that was synced (not uploaded);

$ phttp https://localhost/pulp/api/v2/repositories/zoo/search/units/ criteria:='{"fields": {"unit": ["name", "summary", "size", "group", "sourcerpm"]}, "type_ids": ["rpm"], "filters": {"unit": {"name": "kangaroo"}}}'                                                                                                                                                                                                

[
    {
        "_id": {
            "$oid": "5913618307425414f84ec20f"
        },
        "created": "2017-05-10T18:52:51Z",
        "metadata": {
            "_content_type_id": "rpm",
            "_id": "460666b7-6c66-4e8e-80a1-da0af2374ebd",
            "group": "Internet/Applications",
            "name": "kangaroo",
            "size": 2467,
            "sourcerpm": "kangaroo-0.2-1.src.rpm",
            "summary": "A dummy package of kangaroo"
        },
        "repo_id": "zoo",
        "unit_id": "460666b7-6c66-4e8e-80a1-da0af2374ebd",
        "unit_type_id": "rpm",
        "updated": "2017-05-10T18:52:51Z"
    }
]
Actions #3

Updated by daviddavis over 5 years ago

  • Description updated (diff)
Actions #4

Updated by amacdona@redhat.com over 5 years ago

  • Triaged changed from No to Yes
Actions #5

Updated by mhrivnak over 5 years ago

This was a similar issue. No idea if they're related: https://pulp.plan.io/issues/2365

Actions #6

Updated by daviddavis over 5 years ago

I cleaned my database and tried uploading the kangaroo package from comment 2:

wget https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/kangaroo-0.2-1.noarch.rpm
pulp-admin rpm repo uploads rpm --file kangaroo-0.2-1.noarch.rpm --repo-id zoo
phttp https://localhost/pulp/api/v2/repositories/zoo/search/units/ criteria:='{"fields": {"unit": ["name", "summary", "size", "group", "sourcerpm"]}, "type_ids": ["rpm"], "filters": {"unit": {"name": "kangaroo"}}}' 

Result:

[
    {
        "_id": {
            "$oid": "5915d12607425414f84ec27c"
        },
        "created": "2017-05-12T15:13:42Z",
        "metadata": {
            "_content_type_id": "rpm",
            "_id": "230fd66e-abca-429f-b651-e7ff9c59ab96",
            "name": "kangaroo"
        },
        "repo_id": "zoo",
        "unit_id": "230fd66e-abca-429f-b651-e7ff9c59ab96",
        "unit_type_id": "rpm",
        "updated": "2017-05-12T15:13:42Z"
    }
]
Actions #7

Updated by daviddavis over 5 years ago

Digging into the code a bit, it looks like we parse the headers for uploaded content in yum upload importer[0]. These headers seem to contain the information we need:

In [16]: headers['group']
Out[16]: 'Internet/Applications'

In [17]: headers['summary']
Out[17]: 'A dummy package of dolphin'

In [18]: headers['name']
Out[18]: 'dolphin'

In [19]: headers['size']
Out[19]: 42L

In [20]: headers['sourcerpm']
Out[20]: 'dolphin-3.10.232-1.src.rpm'

However, it appears looking at say the CLI, Pulp doesn't care about these values[1].

Given that a user reported this though, I think we should maybe try to (at the very least) store these values in the database and return them in the API for uploaded rpms (we already do this for synced rpms). I am not sure what sort of implications that has though.

[0] https://github.com/pulp/pulp_rpm/blob/master/plugins/pulp_rpm/plugins/importers/yum/upload.py#L482
[1] https://github.com/pulp/pulp_rpm/blob/master/extensions_admin/pulp_rpm/extensions/admin/contents.py#L31-L33

Actions #8

Updated by mhrivnak over 5 years ago

  • Sprint/Milestone set to 38
Actions #9

Updated by fdobrovo over 5 years ago

  • Status changed from NEW to ASSIGNED
Actions #10

Updated by fdobrovo over 5 years ago

  • Assignee set to fdobrovo
Actions #11

Updated by mhrivnak over 5 years ago

  • Sprint/Milestone changed from 38 to 39
Actions #12

Updated by ttereshc over 5 years ago

  • Status changed from ASSIGNED to NEW
  • Assignee deleted (fdobrovo)
Actions #14

Updated by fdobrovo over 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fdobrovo
Actions #15

Updated by mhrivnak over 5 years ago

  • Sprint/Milestone changed from 39 to 40
Actions #16

Updated by mhrivnak over 5 years ago

  • Sprint/Milestone changed from 40 to 41
Actions #17

Updated by fdobrovo over 5 years ago

  • Status changed from ASSIGNED to POST
Actions #18

Updated by mhrivnak over 5 years ago

  • Sprint/Milestone changed from 41 to 42

Added by fdobrovo over 5 years ago

Revision 62db898a

Fix extracting metadata for rpm/srpm upload

fixes #2754 https://pulp.plan.io/issues/2754

Actions #19

Updated by fdobrovo over 5 years ago

  • Status changed from POST to MODIFIED
Actions #20

Updated by pcreech over 5 years ago

  • Platform Release set to 2.14.0
Actions #21

Updated by pcreech over 5 years ago

  • Status changed from MODIFIED to 5
Actions #22

Updated by pthomas@redhat.com over 5 years ago

The automated test for this issue is passing.

Actions #23

Updated by pcreech over 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #24

Updated by bmbouter almost 5 years ago

  • Sprint set to Sprint 23
Actions #25

Updated by bmbouter almost 5 years ago

  • Sprint/Milestone deleted (42)
Actions #26

Updated by bmbouter almost 4 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF