https://pulp.plan.io/https://pulp.plan.io/favicon.ico2018-10-04T13:25:49ZPulpRPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=312152018-10-04T13:25:49Zipanova@redhat.comipanova@redhat.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/31215/diff?detail_id=32061">diff</a>)</li></ul> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=313052018-10-09T14:39:23ZCodeHeeler
<ul><li><strong>Triaged</strong> changed from <i>No</i> to <i>Yes</i></li></ul> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=374882019-04-12T20:20:29Zbmbouterbmbouter@redhat.com
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>CLOSED - WONTFIX</i></li></ul><p>Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" class="external">developer mailing list</a>.</p> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=377322019-04-15T20:08:31Zbmbouterbmbouter@redhat.com
<ul><li><strong>Tags</strong> <i>Pulp 2</i> added</li></ul> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=438252019-05-16T17:53:21Zggainey
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-9 priority-6 priority-default closed" href="/issues/4798">Issue #4798</a>: Rpm file uploaded/published successfully but not accessible</i> added</li></ul> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=438282019-05-16T17:57:00Zggainey
<ul><li><strong>Status</strong> changed from <i>CLOSED - WONTFIX</i> to <i>ASSIGNED</i></li><li><strong>Assignee</strong> set to <i>ggainey</i></li><li><strong>Sprint</strong> set to <i>Sprint 53</i></li></ul><p>Reopening, as this appears to be causing Other Issues (see <a class="issue tracker-1 status-9 priority-6 priority-default closed" title="Issue: Rpm file uploaded/published successfully but not accessible (CLOSED - WONTFIX)" href="https://pulp.plan.io/issues/4798">#4798</a>)</p> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=443522019-06-04T14:48:02Zamacdona@redhat.comaustin@redhat.com
<ul><li><strong>Sprint</strong> changed from <i>Sprint 53</i> to <i>Sprint 54</i></li></ul> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=446162019-06-11T20:31:56Zggainey
<ul></ul><p>[notes from investigating, before I forget everything]</p>
<p>After a bunch of experimentation, I've discovered there are two problems with upload.</p>
<p>The first is exactly as explained in the description. An uploaded RPM is stored using its upload_id instead of its name, and that causes problems at rsync-distribution-time.</p>
<p>As a specific example - uploading 'rpm-with-non-ascii-1-1.fc25.noarch.rpm' leaves us with a path like</p>
<p><code>/var/lib/pulp/content/units/rpm/45/092320c072b594d5951ef9e8663c8b7ccfd529f1659d33c7087ec29606a4a4/989f7cbd-e61c-4183-92b6-8b31e9ded37e</code></p>
<p>instead of</p>
<p><code>/var/lib/pulp/content/units/rpm/59/e9b952c2a8ab9442864adf0379f5bcc002ae219eb679d1635cce6c08e6d54d/rpm-with-non-ascii-1-1.fc25.noarch.rpm</code></p>
<p>The second problem is when you attempt to upload an RPM to a deferred-download repository, where a) the repository has been sync'd at least once (so we know its RPMs), b) the RPM being uploaded is 'known' to the repository, and c) the feed for the deferred-repository is unreachable or otherwise broken - uploading an RPM 'directly' to get around the broken-feed problem. (see <a class="issue tracker-1 status-9 priority-6 priority-default closed" title="Issue: Rpm file uploaded/published successfully but not accessible (CLOSED - WONTFIX)" href="https://pulp.plan.io/issues/4798">#4798</a>)</p>
<p>Generally, uploading an RPM with the same NEVRA as one already in the repository won't do anything - if you want to replace a NEVRA, delete the old one first, then upload the new. If a broken feed means you can't get the 'original' RPM, then fix the feed and get it 'the right way'. However, as noted in <a class="issue tracker-1 status-9 priority-6 priority-default closed" title="Issue: Rpm file uploaded/published successfully but not accessible (CLOSED - WONTFIX)" href="https://pulp.plan.io/issues/4798">#4798</a>, <a class="issue tracker-1 status-8 priority-6 priority-default closed" title="Issue: Repo Feed Url Change does not update lazy_content_catalog (CLOSED - NOTABUG)" href="https://pulp.plan.io/issues/4265">#4265</a> prevents us from fixing the feed.</p>
<p>This doesn't mean we should add a special case to teach upload to notice that a feed is broken and allow uploading an existing NEVRA 'just this once'. Rather than complicate this issue with yet another edge-case, we'll fix the upload-filepath problem that breaks the rsync publisher here, and fix the broken-feed-for-deferred-repos problem under <a class="issue tracker-1 status-8 priority-6 priority-default closed" title="Issue: Repo Feed Url Change does not update lazy_content_catalog (CLOSED - NOTABUG)" href="https://pulp.plan.io/issues/4265">#4265</a>.</p> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=446292019-06-12T18:25:21Zggainey
<ul><li><strong>Status</strong> changed from <i>ASSIGNED</i> to <i>CLOSED - WONTFIX</i></li></ul><p>After further investigation, I'm back to closing this as WONTFIX. The original description is correct - RPM-upload does result in the actual RPM binary ending up with its upload-id as the filename, instead of the 'real' filename. However, the only place to see this is under the content/units portion of pulp's tree. The Packages symlink in the repo has the correct name, and points to the correct file; this works after export as well (tested rsycn and ISO exports)</p>
<p>As an example - I created a bare repo, <code>empty</code> , and uploaded a single RPM into it, <code>rpm-with-non-ascii-1-1.fc25.noarch.rpm</code></p>
<p>pulp-admin rpm repo create --repo-id empty --relative-url empty<br>
pulp-admin rpm repo uploads rpm --file rpm-with-non-ascii-1-1.fc25.noarch.rpm --repo-id empty<br>
pulp-admin rpm repo publish run --repo-id empty</p>
<p>It clearly has the 'wrong' name on the filesystem:</p>
<pre><code>[vagrant@pulp2 ~]$ sudo tree /var/lib/pulp
/var/lib/pulp
├── celery
├── content
│ └── units
│ └── rpm
│ └── 45
│ └── 092320c072b594d5951ef9e8663c8b7ccfd529f1659d33c7087ec29606a4a4
│ └── aab88552-2c46-4b5d-a57d-cf134b4340f3
├── nodes
│ └── published
</code></pre>
<p>But the link in the published repo is named correctly and points to the right place:</p>
<pre><code>├── published
│ └── yum
│ ├── http
│ ├── https
...
│ │ └── repos
│ │ ├── empty -> /var/lib/pulp/published/yum/master/yum_distributor/empty/1560350550.08
│ │ └── listing
│ └── master
...
│ └── yum_distributor
│ └── empty
│ └── 1560350550.08
│ ├── Packages
│ │ └── r
│ │ └── rpm-with-non-ascii-1-1.fc25.noarch.rpm -> /var/lib/pulp/content/units/rpm/45/092320c072b594d5951ef9e8663c8b7ccfd529f1659d33c7087ec29606a4a4/aab88552-2c46-4b5d-a57d-cf134b4340f3
│ └── repodata
...
</code></pre>
<p>And I can retrieve the RPM using its correct name:</p>
<pre><code>[vagrant@pulp2 ~]$ curl -L -k -o rpm-with-non-ascii.rpm https://localhost/pulp/repos/empty/Packages/r/rpm-with-non-ascii-1-1.fc25.noarch.rpm
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6432 100 6432 0 0 216k 0 --:--:-- --:--:-- --:--:-- 216k
[vagrant@pulp2 ~]$ file rpm-with-non-ascii.rpm
rpm-with-non-ascii.rpm: RPM v3.0 bin i386/x86_64 rpm-with-non-ascii-1-1.fc25
[vagrant@pulp2 ~]$
</code></pre>
<p>And when I export this repo:</p>
<pre><code>[vagrant@pulp2 ~]$ http --verify no --auth admin:admin POST https://localhost/pulp/api/v2/repositories/empty/distributors/ < rsync_distributor_config.json
[vagrant@pulp2 ~]$ http --verify no --auth admin:admin POST https://localhost/pulp/api/v2/repositories/empty/actions/publish/ < publish_rsync.json
</code></pre>
<p>The resulting rsync'd tree on the destination machine is a correct repository:</p>
<pre><code>~/RSYNC $ tree
.
├── content
│ └── units
│ └── rpm
│ └── 45
│ └── 092320c072b594d5951ef9e8663c8b7ccfd529f1659d33c7087ec29606a4a4
│ └── aab88552-2c46-4b5d-a57d-cf134b4340f3
└── empty
├── Packages
│ └── r
│ └── rpm-with-non-ascii-1-1.fc25.noarch.rpm -> ../../../content/units/rpm/45/092320c072b594d5951ef9e8663c8b7ccfd529f1659d33c7087ec29606a4a4/aab88552-2c46-4b5d-a57d-cf134b4340f3
└── repodata
├── 00e8c4132473d5f6e823d5b6b7c4850ec2eec5224f053a00d62cf73447fba90d-primary.xml.gz
├── 474a6fd724269774139856dadc69b675a47d35a4c21b666ae4d0244b64b391fe-updateinfo.xml.gz
├── a27718cc28ec6d71432e0ef3e6da544b7f9d93f6bb7d0a55aacd592d03144b70-comps.xml
├── b72a89d18f6871e7009bf833a71d461e2ec766663991e4f8601fd3e73a386728-other.xml.gz
├── bc7b0ed20aefa013eb348dfb191c54fcdf5781c71c21ecb027b50a48b409b328-filelists.xml.gz
└── repomd.xml
9 directories, 8 files
~/RSYNC $
</code></pre>
<p>All of the above being said - it's possible I'm missing where something is actually broken - please re-open with an explanation if so!</p> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=458562019-07-19T13:41:19Zggainey
<ul><li><strong>Status</strong> changed from <i>CLOSED - WONTFIX</i> to <i>NEW</i></li><li><strong>Sprint</strong> deleted (<del><i>Sprint 54</i></del>)</li></ul><p>DELIVERY-5013 shows a use-case where this behavior is causing pain for our users - re-opening for consideration for future sprint</p> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=467412019-08-19T19:56:53Zggainey
<ul><li><strong>Status</strong> changed from <i>NEW</i> to <i>POST</i></li><li><strong>Groomed</strong> changed from <i>No</i> to <i>Yes</i></li><li><strong>Sprint Candidate</strong> changed from <i>No</i> to <i>Yes</i></li><li><strong>Sprint</strong> set to <i>Sprint 57</i></li></ul><p>Figured out how to fix this.</p>
<p><a href="https://github.com/pulp/pulp_rpm/pull/1421" class="external">https://github.com/pulp/pulp_rpm/pull/1421</a></p> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=470292019-08-26T14:32:08Zrchan
<ul><li><strong>Sprint</strong> changed from <i>Sprint 57</i> to <i>Sprint 58</i></li></ul> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=472552019-09-05T15:50:37Zggainey
<ul><li><strong>Status</strong> changed from <i>POST</i> to <i>MODIFIED</i></li></ul><p>Applied in changeset <a class="changeset" title="Teach upload to use RPM filename instead of upload-id closes #4059 https://pulp.plan.io/issues/4059" href="https://pulp.plan.io/projects/pulp_rpm/repository/9/revisions/5154cebd154558bea46ee90e15e93454052812a2">5154cebd154558bea46ee90e15e93454052812a2</a>.</p> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=473332019-09-09T20:23:04Zkersom
<ul><li><strong>Related to</strong> <i><a class="issue tracker-1 status-11 priority-6 priority-default closed" href="/issues/5419">Issue #5419</a>: Regression - rpm_basefilename tests failures</i> added</li></ul> RPM Support - Issue #4059: During rpm upload filename should be preserved in the storage path https://pulp.plan.io/issues/4059?journal_id=480632019-10-01T13:39:46Zdalleydalley@redhat.com
<ul><li><strong>Status</strong> changed from <i>MODIFIED</i> to <i>CLOSED - CURRENTRELEASE</i></li><li><strong>Platform Release</strong> set to <i>2.21.0</i></li></ul>