Order of data in PULP_MANIFEST returned by Pulp is different from feed url
Order of data in PULP_MANIFEST returned by Pulp is different than what is provided by the synced repo.
1. Create a repository
2. Create a file remote - https://repos.fedorapeople.org/pulp/pulp/fixtures/file/
3. Sync the file remote
4. Create a file publisher
5. Create a publication
6. Create a distribution from the publication
7. Fetch PULP_MANIFEST
PULP_MANIFEST provided by feed url
PULP_MANIFEST downloaded from Pulp
'3.iso, ddc5a9ac99a0cb546cce44be3da447c6e591df8d4860c592f3f1be6e33b66e62, 1024\n2.iso, 7ab0ad049b044879b03d3bc5acbe4e43c98c359fe52a60475e6611ee55033646, 1024\n1.iso, cbd1d07a63f8ac122b7adf75658fc22f9754796f8bbcd9395f1bcc00bbc6e2d8, 1024\n'
This makes verifying the integrity of PULP_MANIFEST downloaded from Pulp a bit more complex.
#4 Updated by ttereshc about 2 years ago
I don't think it's a bug.
In general, it's not safe to rely on the order of metadata.
E.g. RPM packages in primary.xml can be in different order every time, depends on how createrepo_c handles it, not under Pulp's control.
For PULP_MANIFEST: each row has a specific format - data in a certain order separated by commas: relative_path, checksum, size.
The order of rows is not guaranteed to be preserved.
If we decide to publish in incremental way at some point in the future (adding metadata to the existing file), there will be no good way to preserve the order, even if we want to.
If this is needed for test purposes: split by newline and sort.
The inconsistency I can see which is potentially not good and inconvenient is that Pulp produces additional spaces between comma-separated values.
#5 Updated by daviddavis about 2 years ago
I don't think the ordering is a bug either but I find it strange that we sort the files deterministically by when they are created. If we sort at all it should be by filename but we sort by created as a way to eliminate duplicates in the manifest (see ).
I agree that the space after commas should be fixed. Also, users shouldn't rely on the ordering of the manifest file--we should probably focus our efforts on #4028 instead.
#8 Updated by email@example.com about 2 years ago
- Triaged changed from No to Yes
I think this is a problem isolated to the pulp_file plugin. Most plugins implement the API of some existing ecosystem, but pulp_file creates its own (the PULP_MANIFEST).
Since this API does not exist elsewhere, a section needs to be added to the pulp_file docs to explain what this manifest is, how it is structured (and that it is not ordered).
Please register to edit this issue