Story #5008
closedAs a user, I can enforce content specific uniqueness constaints in a repo version
100%
Description
RemoveDuplicates stage provides the opportunity to enforce uniqueness constraints for content at the sync time.
The duplicate problem can be encountered at any time content is added to a repository, not only at sync time.
E.g. Content was uploaded, or content was synced as a part of other repo, and now it's added/copied to a new repository.
It would be good if some check/validation happened for any added content, e.g. at this stage
Examples for where uniqueness might be needed in a repo version:
- only one content unit with a certain characteristics should be present in a repo (only one advisory with the same id, only one module_defaults for a module, etc)
This issue was discussed on this thread: https://www.redhat.com/archives/pulp-dev/2019-May/msg00061.html
Solution¶
Plugins:
On the plugin content model define a repo_key
- one or more fields which must be unique within a repo version.
Pulpcore:
Check uniqueness of the repo_key
for each added content type at the repository version creation time https://github.com/pulp/pulpcore/blob/aef490e201f89fc005ba3239fda3a79c05e28fd7/pulpcore/app/models/repository.py#L343.
Whether it's sync, copy, or upload, repo_key
uniqueness will be ensured if core/plugin devs use with RepositoryVersion.create(...)
context manager.
Related issues
Updated by ttereshc over 5 years ago
- Related to Task #4028: Prevent duplicate files in repositories added
Updated by ttereshc over 5 years ago
- Related to Story #4898: As a user, no "duplicated" content is present in a repo version added
Updated by bmbouter over 5 years ago
This all looks good. Should we convert this to a story? To me in the changelog this would make the most sense as a new feature in the pulpcore-plugin package. What do you think?
Updated by ttereshc over 5 years ago
- Tracker changed from Issue to Story
- Subject changed from No way to enforce content specific uniqueness constaints in a repo version to As a user, I can enforce content specific uniqueness constaints in a repo version
- % Done set to 0
Updated by bmbouter over 5 years ago
- Groomed changed from No to Yes
- Sprint Candidate changed from No to Yes
This looks good to me. Thank you for recapping ttereshc!
Updated by bmbouter over 5 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to bmbouter
Updated by bmbouter about 5 years ago
- Status changed from ASSIGNED to POST
PR available at: https://github.com/pulp/pulpcore/pull/331
Added by bmbouter about 5 years ago
Updated by bmbouter about 5 years ago
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
Applied in changeset pulpcore|b41a0923183c4316401349fb4f823b76433e9ed9.
Updated by bmbouter about 5 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Add repo_key support for
add_content
The
Content
model now supports arepo_key
attribute which defaults to (). Plugin writers can specify this, and the tuple field names will automatically replace another unit of the same type with these field names.Required PR: https://github.com/pulp/pulp_file/pull/290
https://pulp.plan.io/issues/5008 closes #5008