During rpm upload filename should be preserved in the storage path
As of now if rpm is uploaded the storage path will look like:
"_storage_path" : "/var/lib/pulp/content/units/rpm/d8/35d09b8cb744866dfd913f1f4fa64700683c1199351350c770e67760ce03a0/73744eb6-01df-4386-a647-2fe64cc95c8e",
Instead of the filename, the upload_id is used during the construction of storage path.
This will lead to the rsync publish distributor to publish files without trailing filename since it uses the _storage_path
#3 Updated by bmbouter over 1 year ago
- Status changed from NEW to CLOSED - WONTFIX
Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.
#8 Updated by ggainey over 1 year ago
[notes from investigating, before I forget everything]
After a bunch of experimentation, I've discovered there are two problems with upload.
The first is exactly as explained in the description. An uploaded RPM is stored using its upload_id instead of its name, and that causes problems at rsync-distribution-time.
As a specific example - uploading 'rpm-with-non-ascii-1-1.fc25.noarch.rpm' leaves us with a path like
The second problem is when you attempt to upload an RPM to a deferred-download repository, where a) the repository has been sync'd at least once (so we know its RPMs), b) the RPM being uploaded is 'known' to the repository, and c) the feed for the deferred-repository is unreachable or otherwise broken - uploading an RPM 'directly' to get around the broken-feed problem. (see #4798)
Generally, uploading an RPM with the same NEVRA as one already in the repository won't do anything - if you want to replace a NEVRA, delete the old one first, then upload the new. If a broken feed means you can't get the 'original' RPM, then fix the feed and get it 'the right way'. However, as noted in #4798, #4265 prevents us from fixing the feed.
This doesn't mean we should add a special case to teach upload to notice that a feed is broken and allow uploading an existing NEVRA 'just this once'. Rather than complicate this issue with yet another edge-case, we'll fix the upload-filepath problem that breaks the rsync publisher here, and fix the broken-feed-for-deferred-repos problem under #4265.
#9 Updated by ggainey over 1 year ago
- Status changed from ASSIGNED to CLOSED - WONTFIX
After further investigation, I'm back to closing this as WONTFIX. The original description is correct - RPM-upload does result in the actual RPM binary ending up with its upload-id as the filename, instead of the 'real' filename. However, the only place to see this is under the content/units portion of pulp's tree. The Packages symlink in the repo has the correct name, and points to the correct file; this works after export as well (tested rsycn and ISO exports)
As an example - I created a bare repo,
empty , and uploaded a single RPM into it,
pulp-admin rpm repo create --repo-id empty --relative-url empty
pulp-admin rpm repo uploads rpm --file rpm-with-non-ascii-1-1.fc25.noarch.rpm --repo-id empty
pulp-admin rpm repo publish run --repo-id empty
It clearly has the 'wrong' name on the filesystem:
[vagrant@pulp2 ~]$ sudo tree /var/lib/pulp /var/lib/pulp ├── celery ├── content │ └── units │ └── rpm │ └── 45 │ └── 092320c072b594d5951ef9e8663c8b7ccfd529f1659d33c7087ec29606a4a4 │ └── aab88552-2c46-4b5d-a57d-cf134b4340f3 ├── nodes │ └── published
But the link in the published repo is named correctly and points to the right place:
├── published │ └── yum │ ├── http │ ├── https ... │ │ └── repos │ │ ├── empty -> /var/lib/pulp/published/yum/master/yum_distributor/empty/1560350550.08 │ │ └── listing │ └── master ... │ └── yum_distributor │ └── empty │ └── 1560350550.08 │ ├── Packages │ │ └── r │ │ └── rpm-with-non-ascii-1-1.fc25.noarch.rpm -> /var/lib/pulp/content/units/rpm/45/092320c072b594d5951ef9e8663c8b7ccfd529f1659d33c7087ec29606a4a4/aab88552-2c46-4b5d-a57d-cf134b4340f3 │ └── repodata ...
And I can retrieve the RPM using its correct name:
[vagrant@pulp2 ~]$ curl -L -k -o rpm-with-non-ascii.rpm https://localhost/pulp/repos/empty/Packages/r/rpm-with-non-ascii-1-1.fc25.noarch.rpm % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 6432 100 6432 0 0 216k 0 --:--:-- --:--:-- --:--:-- 216k [vagrant@pulp2 ~]$ file rpm-with-non-ascii.rpm rpm-with-non-ascii.rpm: RPM v3.0 bin i386/x86_64 rpm-with-non-ascii-1-1.fc25 [vagrant@pulp2 ~]$
And when I export this repo:
[vagrant@pulp2 ~]$ http --verify no --auth admin:admin POST https://localhost/pulp/api/v2/repositories/empty/distributors/ < rsync_distributor_config.json [vagrant@pulp2 ~]$ http --verify no --auth admin:admin POST https://localhost/pulp/api/v2/repositories/empty/actions/publish/ < publish_rsync.json
The resulting rsync'd tree on the destination machine is a correct repository:
~/RSYNC $ tree . ├── content │ └── units │ └── rpm │ └── 45 │ └── 092320c072b594d5951ef9e8663c8b7ccfd529f1659d33c7087ec29606a4a4 │ └── aab88552-2c46-4b5d-a57d-cf134b4340f3 └── empty ├── Packages │ └── r │ └── rpm-with-non-ascii-1-1.fc25.noarch.rpm -> ../../../content/units/rpm/45/092320c072b594d5951ef9e8663c8b7ccfd529f1659d33c7087ec29606a4a4/aab88552-2c46-4b5d-a57d-cf134b4340f3 └── repodata ├── 00e8c4132473d5f6e823d5b6b7c4850ec2eec5224f053a00d62cf73447fba90d-primary.xml.gz ├── 474a6fd724269774139856dadc69b675a47d35a4c21b666ae4d0244b64b391fe-updateinfo.xml.gz ├── a27718cc28ec6d71432e0ef3e6da544b7f9d93f6bb7d0a55aacd592d03144b70-comps.xml ├── b72a89d18f6871e7009bf833a71d461e2ec766663991e4f8601fd3e73a386728-other.xml.gz ├── bc7b0ed20aefa013eb348dfb191c54fcdf5781c71c21ecb027b50a48b409b328-filelists.xml.gz └── repomd.xml 9 directories, 8 files ~/RSYNC $
All of the above being said - it's possible I'm missing where something is actually broken - please re-open with an explanation if so!
Please register to edit this issue