Issue #2754
closedRPM uploads appear to be missing metadata information
Description
Specifically, the fields Size, Summary, Source RPM, and Group appear to be missing.
Steps:
$ wget ftp://195.220.108.108/linux/fedora-secondary/development/rawhide/Everything/aarch64/os/Packages/v/vim-enhanced-8.0.596-1.fc27.aarch64.rpm
$ rpm -qip vim-enhanced-8.0.596-1.fc27.aarch64.rpm
...
Group : Applications/Editors
Size : 3028853
...
Source RPM : vim-8.0.596-1.fc27.src.rpm
...
Summary : A version of the VIM editor which includes recent enhancements
...
$ pulp-admin rpm repo uploads rpm --file vim-enhanced-8.0.596-1.fc27.aarch64.rpm --repo-id zoo
$ phttp https://localhost/pulp/api/v2/repositories/zoo/search/units/ criteria:='{"fields": {"unit": ["name", "summary", "size", "group", "sourcerpm"]}, "type_ids": ["rpm"], "filters": {"unit": {"name"
: "vim-enhanced"}}}'
[
{
"_id": {
"$oid": "5915aac507425414f84ec276"
},
"created": "2017-05-12T12:29:57Z",
"metadata": {
"_content_type_id": "rpm",
"_id": "939240d9-2071-451d-8749-c10cac057f31",
"name": "vim-enhanced"
},
"repo_id": "zoo",
"unit_id": "939240d9-2071-451d-8749-c10cac057f31",
"unit_type_id": "rpm",
"updated": "2017-05-12T12:29:57Z"
}
]
I checked the database too and the fields aren't there.
Since this is also a Katello issue, we should notify them if they need to be setting particular metadata fields on their import upload request too.
Updated by daviddavis about 7 years ago
- Description updated (diff)
Just for comparison, here's another repo that was synced (not uploaded);
$ phttp https://localhost/pulp/api/v2/repositories/zoo/search/units/ criteria:='{"fields": {"unit": ["name", "summary", "size", "group", "sourcerpm"]}, "type_ids": ["rpm"], "filters": {"unit": {"name": "kangaroo"}}}'
[
{
"_id": {
"$oid": "5913618307425414f84ec20f"
},
"created": "2017-05-10T18:52:51Z",
"metadata": {
"_content_type_id": "rpm",
"_id": "460666b7-6c66-4e8e-80a1-da0af2374ebd",
"group": "Internet/Applications",
"name": "kangaroo",
"size": 2467,
"sourcerpm": "kangaroo-0.2-1.src.rpm",
"summary": "A dummy package of kangaroo"
},
"repo_id": "zoo",
"unit_id": "460666b7-6c66-4e8e-80a1-da0af2374ebd",
"unit_type_id": "rpm",
"updated": "2017-05-10T18:52:51Z"
}
]
Updated by mhrivnak about 7 years ago
This was a similar issue. No idea if they're related: https://pulp.plan.io/issues/2365
Updated by daviddavis about 7 years ago
I cleaned my database and tried uploading the kangaroo package from comment 2:
wget https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/kangaroo-0.2-1.noarch.rpm
pulp-admin rpm repo uploads rpm --file kangaroo-0.2-1.noarch.rpm --repo-id zoo
phttp https://localhost/pulp/api/v2/repositories/zoo/search/units/ criteria:='{"fields": {"unit": ["name", "summary", "size", "group", "sourcerpm"]}, "type_ids": ["rpm"], "filters": {"unit": {"name": "kangaroo"}}}'
Result:
[
{
"_id": {
"$oid": "5915d12607425414f84ec27c"
},
"created": "2017-05-12T15:13:42Z",
"metadata": {
"_content_type_id": "rpm",
"_id": "230fd66e-abca-429f-b651-e7ff9c59ab96",
"name": "kangaroo"
},
"repo_id": "zoo",
"unit_id": "230fd66e-abca-429f-b651-e7ff9c59ab96",
"unit_type_id": "rpm",
"updated": "2017-05-12T15:13:42Z"
}
]
Updated by daviddavis about 7 years ago
Digging into the code a bit, it looks like we parse the headers for uploaded content in yum upload importer[0]. These headers seem to contain the information we need:
In [16]: headers['group']
Out[16]: 'Internet/Applications'
In [17]: headers['summary']
Out[17]: 'A dummy package of dolphin'
In [18]: headers['name']
Out[18]: 'dolphin'
In [19]: headers['size']
Out[19]: 42L
In [20]: headers['sourcerpm']
Out[20]: 'dolphin-3.10.232-1.src.rpm'
However, it appears looking at say the CLI, Pulp doesn't care about these values[1].
Given that a user reported this though, I think we should maybe try to (at the very least) store these values in the database and return them in the API for uploaded rpms (we already do this for synced rpms). I am not sure what sort of implications that has though.
[0] https://github.com/pulp/pulp_rpm/blob/master/plugins/pulp_rpm/plugins/importers/yum/upload.py#L482
[1] https://github.com/pulp/pulp_rpm/blob/master/extensions_admin/pulp_rpm/extensions/admin/contents.py#L31-L33
Updated by ttereshc about 7 years ago
- Status changed from ASSIGNED to NEW
- Assignee deleted (
fdobrovo)
Updated by fdobrovo about 7 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to fdobrovo
Updated by fdobrovo about 7 years ago
- Status changed from ASSIGNED to POST
Added by fdobrovo about 7 years ago
Updated by fdobrovo about 7 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulp_rpm:62db898a81679b40e3cc3a48af9f3b830f4521cb.
Updated by pcreech almost 7 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
Fix extracting metadata for rpm/srpm upload
fixes #2754 https://pulp.plan.io/issues/2754