Project

Profile

Help

Maintenance: Planio will be observing a scheduled maintenance window this Sunday, November 10, 2024 from 20:00 UTC until 21:00 UTC to perform important network maintenance in our primary data center. Your Planio account will be unavailable for a few minutes during this maintenance window.

Story #4458

closed

Story #5114: [Epic] Publish features

As a user, I can configure which checksum algorithm to use when creating metadata

Added by ttereshc over 5 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Katello
Sprint:
Sprint 70
Quarter:

Description

Support metadata_checksum_type and package_checksum_type options at publish time.
These are one-time options and are passed as a parameter for a publication creation call.
metadata_checksum_type affects all the repodata, including primary.xml, repomd.xml, etc.
package_checksum_type affects package checksum type in all repo metadata files.

Origin of checksum types and their precedence (at the top is the one which is used first if present and available):
- configuration at publish time
- a remote repo (store the checksum type from the remote source on the repository version model)
- Pulp default (sha256)

DNF supports different package checksum types in one repo.

Publication creation should never fail due to checksum type configuration. At least one checksum type is always available for a package and it should be used if the requested one is not available.


Related issues

Related to RPM Support - Test #4469: Test checksum algorithm when publishingCLOSED - COMPLETEActions
Actions #1

Updated by ttereshc over 5 years ago

  • Description updated (diff)
Actions #2

Updated by ttereshc over 5 years ago

Everything should be easy and without any issues when every package in a repo is downloaded. This is the case for:

  • immediate download policy, after sync is performed
  • uploaded packages
  • already downloaded packages copied from other repos

Other cases:
1. On_demand repo, no packages from other sources, first sync from the remote source.

  • If the requested checksum type is different from what is available, publish with available checksum type + log a warning that the requested checksum type can't be used

2. Mixture of packages from different on_demand repos

  • If on_demand repos have different checksum types, there is no way out -> fail

3. Mixture of downloaded content and in_demand.

  • If on_demand content has the common checksum type, it should be used, otherwise fail.

4. On_demand repo, Nth sync, a checksum type in a remote repo changed

  • Not sure what to do with it. We can publish using old checksum type but I guess expectation would be that the new checksum type is used.

Concerns:
- we don't have info about remote repo checksum type, we have checksum type info for each package in a RemoteArtifact.
- does it mean that we should inspect all RemoreArtifacts to figure out what the common checksum type is for on_demand content?

Actions #3

Updated by ipanova@redhat.com over 5 years ago

Case 1.
I wonder if we could just fail the publish with the helpful explanation message? This way we could be sure this kind of incompatibility is acknowledged by user and he'd update the checksum type or resync the repo with immediate policy.

Case 4.
I don't think we can somehow possibly be aware of the changes happened remotely unless we resync. We publish what we have synced down locally.

Actions #4

Updated by kersom over 5 years ago

  • Related to Test #4469: Test checksum algorithm when publishing added
Actions #5

Updated by jsherril@redhat.com over 5 years ago

My two cents:

  • checksum types on metadata are not a security feature
  • Publishing a repository should work all the time, even if a consistent checksum type is not used
  • Publishing should never fail (unless the user explicitly declares they want it to fail unless some condition is met)

In my experience, Katello's end users are met with pure confusion when any of the above error scenarios occur, which results in a failure. They didn't make any of the decisions which led to the failure, so they have no idea how to resolve it. Its almost always no fault of their own, and it results in a bad user experience.

My recommendation:

1. We have a setting 'metadata_checksum' which controls which checksum types the metadata are recorded with (in repomd.xml) - Optional
2. We have another setting 'preferred_package_checksum_type' (or something like that), which indicates the preference of the checksum type for the user.
3. If desired, we have 3rd setting 'strict_package_checksum_type', which does fail in the cases above, this seems less useful.

Going to your scenarios above:

2. If 'preferred_package_checksum_type' is set, generate with mixed checksum types, log a warning. If its not set, generate with whatever is upstream

3. Similar to 2

4. Yes, i'd expect the new checksum type

Your concerns:

Concerns:
- we don't have info about remote repo checksum type, we have checksum type info for each package in a RemoteArtifact.
Could this be stored read-only in the remote, and the user could then pass that to the publisher?

Actions #6

Updated by bmbouter over 5 years ago

  • Tags deleted (Pulp 3)
Actions #7

Updated by ttereshc almost 5 years ago

  • Description updated (diff)

Updated the description based on discussions with Katello and DNF.

Actions #8

Updated by ttereshc almost 5 years ago

Actions #9

Updated by CodeHeeler almost 5 years ago

  • Groomed changed from No to Yes
Actions #10

Updated by CodeHeeler almost 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to CodeHeeler
Actions #11

Updated by CodeHeeler almost 5 years ago

  • Sprint set to Sprint 65
Actions #12

Updated by rchan almost 5 years ago

  • Sprint changed from Sprint 65 to Sprint 66
Actions #13

Updated by rchan over 4 years ago

  • Sprint changed from Sprint 66 to Sprint 67
Actions #14

Updated by ttereshc over 4 years ago

  • Sprint/Milestone set to Pulp 3.x RPM (Katello 3.16)
  • Parent issue set to #5114
Actions #15

Updated by rchan over 4 years ago

  • Sprint changed from Sprint 67 to Sprint 68
Actions #16

Updated by rchan over 4 years ago

  • Sprint changed from Sprint 68 to Sprint 69
Actions #17

Updated by CodeHeeler over 4 years ago

  • Status changed from ASSIGNED to NEW
  • Assignee deleted (CodeHeeler)
Actions #18

Updated by CodeHeeler over 4 years ago

  • Sprint Candidate changed from No to Yes
Actions #19

Updated by fao89 over 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fao89
Actions #20

Updated by pulpbot over 4 years ago

  • Status changed from ASSIGNED to POST
Actions #21

Updated by rchan over 4 years ago

  • Sprint changed from Sprint 69 to Sprint 70

Added by Fabricio Aguiar over 4 years ago

Revision 6e453c93 | View on GitHub

Enabled checksum selection when creating metadata

https://pulp.plan.io/issues/4458 closes #4458

Actions #22

Updated by Anonymous over 4 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

Added by Fabricio Aguiar over 4 years ago

Revision 914750af | View on GitHub

Fix publication checksum issue

https://pulp.plan.io/issues/6505 fixes #4458

Actions #24

Updated by dalley over 4 years ago

  • Sprint/Milestone changed from Pulp 3.x RPM (Katello 3.16) to Pulp RPM 3.3.0
Actions #25

Updated by dalley over 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #26

Updated by ggainey over 4 years ago

  • Tags Katello added
  • Tags deleted (Katello-P2)

Also available in: Atom PDF