Story #7502
closedAs a user I want to sync upstream repositories using "Flat Repository Format"
100%
Description
It appears some APT repositories do not store their release files somewhere beneath a dists/
folder, using the repo root (or some arbitrary path) instead.
Example: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/
Apparently this repository can be included in an /etc/apt/sources.list
file as follows (at least on Ubuntu):
deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/ /
There is some Ubuntu documentation hinting at specifying an "exact path" here: http://manpages.ubuntu.com/manpages/xenial/man5/sources.list.5.html (Also see the examples).
Related issues
Updated by quba42 over 4 years ago
Some initial thoughts:
- We need to make sure the solution does not break any existing use cases.
- If there is no unambiguous way to handle these new cases using the existing
distribution
field, then we need a new field/option in our remotes. (use this as a last resort) - We need to make sure we consider every instance of hard coded
dists
path segments in our solution. - The exception is the simple publisher which generally ignores upstream repo structure in favour of its own.
- How should the structured publisher deal with so structured repos? (It probably needs to stay true to its word and retain upstream Release and Packages file placement.)
- We will need to think about fixtures and tests for this.
- I consider this to be a new feature, not a bugfix.
Updated by quba42 over 4 years ago
- Tracker changed from Task to Story
- Subject changed from Support upstream repositories that do not store Release files in a dists/ folder to As a user I want to sync upstream repositories that do not store Release files within /dists/
Updated by quba42 over 4 years ago
It appears Debian has a specification for this as well: https://wiki.debian.org/DebianRepository/Format#Flat_Repository_Format
They refer to this as a "Flat Repository Format".
Updated by quba42 over 4 years ago
- Subject changed from As a user I want to sync upstream repositories that do not store Release files within /dists/ to As a user I want to sync upstream repositories using "Flat Repository Format"
Needs further discussion:
We could add a separate FlatAptRemote
, or try to use the existing AptRemote
.
If we do go for two remotes, a common parent class would probably make sense.
Updated by quba42 about 4 years ago
- Related to Issue #7673: Support synchronization of upstream repos with minimal Release file fields including for flat repos added
Updated by pulpbot about 4 years ago
- Status changed from NEW to POST
Added by ThikaXer about 4 years ago
Updated by quba42 about 4 years ago
Note:
Thought this issue is nearing completion, the example repository in the ticket description won't become synchronizable with this change!
That repository has multiple issues that are independent of it being in flat repository format:
- No
Codename
field in the Release file. - No
Architectures
field in the Release file. (They did add anArchitecture
field that does not contain a valid Debian architecture string...) - No
Components
field in the Release file. - The metadata references packages that don't actually exist. (This can be worked around by using
policy="on_demand"
in the remote).
Since APT is nevertheless willing to consume this repository, we might look into opening separate issues to support these corner cases.
Example repo:
deb http://developer.download.nvidia.com/compute/cuda/repos/debian10/x86_64/ /
Worst repository ever! (For good measure the public key needed to verify the repository is published right there in the repository...)
If we can get this repository to synchronize, we can synchronize anything!
Updated by ThikaXer about 4 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset 80ac6cc3f49aaf3b3b82e4cc8e92a8186e6f2a0e.
Updated by quba42 about 4 years ago
- Sprint/Milestone changed from Katello to 2.8.0
Updated by pulpbot about 4 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Updated by quba42 almost 4 years ago
- File deleted (
cash-for-cars-toowoomba.jpg)
Support flat repository format
fixes #7502 https://pulp.plan.io/issues/7502