Project

Profile

Help

Story #4898

closed

As a user, no "duplicated" content is present in a repo version

Added by ttereshc over 3 years ago. Updated over 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

"Duplicates" are two pieces of content which are not allowed to exist within a single repo version.

E.g.

  • RPMs with the same nevra but different checksum types can't be in a single repo
  • Advisories with the same id can't be in one repo
  • Only one modulemd_defaults per module (name) can be in a single repo

There is a way to prevent that at sync time with RemoveDuplicates stage.
Currently there is no mechanism to prevent it if content is already in Pulp and user copies/adds content to a repo version.

Todos

  • add repo_key to RPMs for its NEVRA (but not checksum)
  • modulemd_default object gets a repo_key
  • remove the RemoveDuplicates stage from the pulp_rpm custom pipeline

Related issues

Related to Pulp - Story #5008: As a user, I can enforce content specific uniqueness constaints in a repo versionCLOSED - CURRENTRELEASEbmbouter

Actions
Related to RPM Support - Story #5084: As a user, after copy or repo version creation there are no advisories with the same idCLOSED - DUPLICATE

Actions
Actions #1

Updated by ttereshc over 3 years ago

  • Description updated (diff)
Actions #2

Updated by ttereshc over 3 years ago

  • Related to Story #5008: As a user, I can enforce content specific uniqueness constaints in a repo version added
Actions #3

Updated by ttereshc about 3 years ago

  • Tags Pulp 3 RPM blocker added
Actions #4

Updated by daviddavis about 3 years ago

  • Related to Story #5084: As a user, after copy or repo version creation there are no advisories with the same id added
Actions #5

Updated by bmbouter almost 3 years ago

If we add repo_key to each of the content units, would we still need the RemoteDuplicates stage?

Actions #6

Updated by ttereshc almost 3 years ago

I don't think we need RemoveDuplicates at all. Should we remove it from the Plugin API? I haven't seen your implementation of repo_key yet but the discussions in the past gave me an impression that it will fully substitute RemoveDuplicates stage.

We can't add a repo_key for Advisories though because we need to merge them. So Advisories will wait for a mechanism we are discussing on the pulp-dev now where a plugin can influence which content is added to a repo version creation.

Actions #7

Updated by bmbouter almost 3 years ago

  • Description updated (diff)

ttereshc wrote:

I don't think we need RemoveDuplicates at all. Should we remove it from the Plugin API? I haven't seen your implementation of repo_key yet but the discussions in the past gave me an impression that it will fully substitute RemoveDuplicates stage.

I agree. This ticket should include removing the RemoveDuplicates stage. I updated a todos section in the body.

We can't add a repo_key for Advisories though because we need to merge them. So Advisories will wait for a mechanism we are discussing on the pulp-dev now where a plugin can influence which content is added to a repo version creation.

I see what you're saying about this being related. I'll reply on that thread some.

Is the Advisory merging unhandled during sync today? I was looking in pulp_rpm and I see it only handles Package and RepoMetadataFile , so repo_key can fully replace RemoveDuplicates stage. Does that sound right to you?

Actions #8

Updated by ttereshc almost 3 years ago

yes, it sounds right. Merging of advisories is not handled at sync at the moment.

Actions #9

Updated by ttereshc almost 3 years ago

  • Assignee set to ttereshc

Added by ttereshc almost 3 years ago

Revision 4e932e58

Ensure no duplicates are present in a repo version.

Also removes one_shot_upload as it's not used anymore.

closes #4898 https://pulp.plan.io/issues/4898

Actions #10

Updated by ttereshc almost 3 years ago

  • Status changed from NEW to MODIFIED
  • % Done changed from 0 to 100
Actions #11

Updated by ttereshc almost 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #12

Updated by bmbouter over 2 years ago

  • Tags deleted (Pulp 3 RPM blocker)

Also available in: Atom PDF