Story #4458
closedStory #5114: [Epic] Publish features
As a user, I can configure which checksum algorithm to use when creating metadata
100%
Description
Support metadata_checksum_type
and package_checksum_type
options at publish time.
These are one-time options and are passed as a parameter for a publication creation call.
metadata_checksum_type
affects all the repodata, including primary.xml, repomd.xml, etc.
package_checksum_type
affects package checksum type in all repo metadata files.
Origin of checksum types and their precedence (at the top is the one which is used first if present and available):
- configuration at publish time
- a remote repo (store the checksum type from the remote source on the repository version model)
- Pulp default (sha256)
DNF supports different package checksum types in one repo.
Publication creation should never fail due to checksum type configuration. At least one checksum type is always available for a package and it should be used if the requested one is not available.
Related issues
Updated by ttereshc almost 6 years ago
Everything should be easy and without any issues when every package in a repo is downloaded. This is the case for:
- immediate download policy, after sync is performed
- uploaded packages
- already downloaded packages copied from other repos
Other cases:
1. On_demand repo, no packages from other sources, first sync from the remote source.
- If the requested checksum type is different from what is available, publish with available checksum type + log a warning that the requested checksum type can't be used
2. Mixture of packages from different on_demand repos
- If on_demand repos have different checksum types, there is no way out -> fail
3. Mixture of downloaded content and in_demand.
- If on_demand content has the common checksum type, it should be used, otherwise fail.
4. On_demand repo, Nth sync, a checksum type in a remote repo changed
- Not sure what to do with it. We can publish using old checksum type but I guess expectation would be that the new checksum type is used.
Concerns:
- we don't have info about remote repo checksum type, we have checksum type info for each package in a RemoteArtifact.
- does it mean that we should inspect all RemoreArtifacts to figure out what the common checksum type is for on_demand content?
Updated by ipanova@redhat.com almost 6 years ago
Case 1.
I wonder if we could just fail the publish with the helpful explanation message? This way we could be sure this kind of incompatibility is acknowledged by user and he'd update the checksum type or resync the repo with immediate policy.
Case 4.
I don't think we can somehow possibly be aware of the changes happened remotely unless we resync. We publish what we have synced down locally.
Updated by kersom almost 6 years ago
- Related to Test #4469: Test checksum algorithm when publishing added
Updated by jsherril@redhat.com almost 6 years ago
My two cents:
- checksum types on metadata are not a security feature
- Publishing a repository should work all the time, even if a consistent checksum type is not used
- Publishing should never fail (unless the user explicitly declares they want it to fail unless some condition is met)
In my experience, Katello's end users are met with pure confusion when any of the above error scenarios occur, which results in a failure. They didn't make any of the decisions which led to the failure, so they have no idea how to resolve it. Its almost always no fault of their own, and it results in a bad user experience.
My recommendation:
1. We have a setting 'metadata_checksum' which controls which checksum types the metadata are recorded with (in repomd.xml) - Optional
2. We have another setting 'preferred_package_checksum_type' (or something like that), which indicates the preference of the checksum type for the user.
3. If desired, we have 3rd setting 'strict_package_checksum_type', which does fail in the cases above, this seems less useful.
Going to your scenarios above:
2. If 'preferred_package_checksum_type' is set, generate with mixed checksum types, log a warning. If its not set, generate with whatever is upstream
3. Similar to 2
4. Yes, i'd expect the new checksum type
Your concerns:
Concerns:
- we don't have info about remote repo checksum type, we have checksum type info for each package in a RemoteArtifact.
Could this be stored read-only in the remote, and the user could then pass that to the publisher?
Updated by ttereshc almost 5 years ago
- Description updated (diff)
Updated the description based on discussions with Katello and DNF.
Updated by CodeHeeler almost 5 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to CodeHeeler
Updated by ttereshc almost 5 years ago
- Sprint/Milestone set to Pulp 3.x RPM (Katello 3.16)
- Parent issue set to #5114
Updated by CodeHeeler over 4 years ago
- Status changed from ASSIGNED to NEW
- Assignee deleted (
CodeHeeler)
Updated by fao89 over 4 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to fao89
Updated by pulpbot over 4 years ago
- Status changed from ASSIGNED to POST
Added by Fabricio Aguiar over 4 years ago
Updated by Anonymous over 4 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset 6e453c934d2b07db6ba58fd97ecb58a21a1b460a.
Updated by pulpbot over 4 years ago
Added by Fabricio Aguiar over 4 years ago
Revision 914750af | View on GitHub
Fix publication checksum issue
Updated by dalley over 4 years ago
- Sprint/Milestone changed from Pulp 3.x RPM (Katello 3.16) to Pulp RPM 3.3.0
Updated by dalley over 4 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Updated by ggainey over 4 years ago
- Tags Katello added
- Tags deleted (
Katello-P2)
Enabled checksum selection when creating metadata
https://pulp.plan.io/issues/4458 closes #4458