Issue #9328
closed
repository metadata not getting published for some repos which have ".." in location_href (points outside of repo)
Status:
CLOSED - CURRENTRELEASE
Description
I created multiple repos and selected default setting but metadata does not get published. Packages directory is missing from published repo. 'manually republishing metadata fixes it but I am afraid it will become stale.
Files
- Project changed from Pulp to RPM Support
- Priority changed from Normal to High
- Sprint set to Sprint 104
@mwaqas@bloombergindustry.com Which versions of pulpcore and pulp_rpm are you using?
- Status changed from NEW to ASSIGNED
- Assignee set to dalley
Confirmed w/ master branch
- Triaged changed from No to Yes
- Subject changed from repository metadata not getting published to repository metadata not getting published for some repos sync from a mirror list
- Subject changed from repository metadata not getting published for some repos sync from a mirror list to repository metadata not getting published for some repos synced from a mirror list
dalley wrote:
@mwaqas@bloombergindustry.com Which versions of pulpcore and pulp_rpm are you using?
rpm -qa |grep -i pulpcore
tfm-rubygem-pulpcore_client-3.14.1-1.el7.noarch
pulpcore-selinux-1.2.4-1.el7.x86_64
python3-pulpcore-3.14.4-1.el7.noarch
rpm -qa |grep -i pulp_rpm
tfm-rubygem-pulp_rpm_client-3.13.3-1.el7.noarch
- Sprint changed from Sprint 104 to Sprint 105
- Subject changed from repository metadata not getting published for some repos synced from a mirror list to repository metadata not getting published for some repos synced from behind Amazon CDN
https://packages.rundeck.com/pagerduty/rundeck/rpm_any/rpm_any/x86_64 is not a mirrorlist repo, so that is not the issue.
But both of these repos are hosted behind the Amazon Cloudfront CDN, which is doing some really crazy and annoying obfuscation (presumably for purposes of DDoS prevention). I don't have proof yet but I suspect that's part of the problem.
edit: I was wrong, that's not the reason. The reason is that these repos use location_href values that point backwards outside of the actual repo. Pulp cannot mirror these paths, so the only solution is to throw an error and tell the user to disable mirroring, so that the location_href values in the metadata can be rewritten inside the path namespace of the distribution.
The solution is to disable "mirror-on-sync". And the "bugfix" here should be to add an hard-error that fails for repositories like this one, and tells the user to disable mirroring.
The "mirror" option makes a complete copy of the metadata and retains the same package layout as the original repo. But because the original layout keeps all of the package outside of the repo itself, mirror can't possibly work. It would be putting the files outside of the distribution base_path and polluting / interfering with the global namespace, and we cannot allow that.
- Status changed from ASSIGNED to POST
- Subject changed from repository metadata not getting published for some repos synced from behind Amazon CDN to repository metadata not getting published for some repos which have ".." in location_href (points outside of repo)
- Sprint/Milestone set to 3.14.4
- Status changed from POST to MODIFIED
- Sprint/Milestone changed from 3.14.4 to 3.16.0
- Copied to Backport #9392: Backport #9328 "repository metadata not getting published for some repos which have ".." in location_href (points outside of repo)" to 3.14.z added
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Also available in: Atom
PDF
.
Dealt with another situation incompatible with mirror syncing
Some repos use location_href values that point outside of the repository. We cannot support this in mirror mode because we cannot write paths outside of the repository namespace.
closes: #9328 https://pulp.plan.io/issues/9328