Support synchronization of upstream repos with minimal Release file fields including for flat repos
The debian repo format specification (https://wiki.debian.org/DebianRepository/Format#A.22Release.22_files) states that: "Servers shall provide [...] (Release files) with at least the following keys:"
- Suite and/or Codename
However, it is also stated that: "Clients may accept [...] Release files without the fields required for servers. They might reject Release files that do not contain at least one of the fields defined herein."
This results in the following conundrum: Should pulp_deb be able to synchronize repositories that do not meet the minimum requirements for servers, but nevertheless work for (most) APT clients?
My personal opinion, is that if APT clients are capable of consuming some upstream repository, then pulp_deb should make a reasonable effort to support that repository structure. In the real world, users will expect to be able to synchronize all repositories that they know to work for their APT clients and people publishing APT repositories will consider their repositories "valid" once they have tested them against an APT client as well.
Currently, pulp_deb does not care about the presence of Date, and Suite fields, so that is not an issue. IMHO, a Release file without the SHA256 field (or some alternate checksum field) does not make any sense and should not be supported. We currently require a Codename field, but I think it would not be so difficult to drop this requirement.
Supporting repos without an Architectures and/or Components field, would be very difficult, since those are (normally) part of the path to the package indecies (Packages files) (and also deeply embedded in our code). However, the exception is repositories using "Flat repository structure": https://wiki.debian.org/DebianRepository/Format#Flat_Repository_Format
Updated by quba42 about 3 years ago
Note: This problem was first identified during work on "Flat repository structure" using the following upstream repository as a test case: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/
Updated by quba42 over 2 years ago
Found another example here: https://pkg.jenkins.io/debian-stable/binary/Release
Added by quba42 about 2 years ago
Make flat repo syncs more robust
With this change flat repo syncs will work even if the upstream Release file contains no "Components" or "Architectures" fields. In addition we are ensuring flat repo syncs do not attempt to sync installer files or packages.