Project

Profile

Help

Issue #4740

Pulpcore doesn't provide a way to guarantee uniqueness in repo versions

Added by daviddavis 6 months ago. Updated 6 months ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
Start date:
Due date:
Severity:
3. High
Version:
Platform Release:
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

Suppose you have two units of content that aren't unique. This could be two packages with the same NEVRA (one signed, one unsigned or two different checksums), two errata with the same id, two files with the same filename, two docker tags with the same name, etc. Let's say that their ids are $CU1 and $CU2. Now you call:

http POST ':24817'$REPO_HREF'versions/' add_content_units:="[\"$CU1\", \"$CU2\"]" 

This causes conflicting content units to be added to a repo version. The problem here is that pulpcore doesn't provide plugin writers with a way to prevent this and ensure uniqueness for repo version content.


Related issues

Duplicates Pulp - Issue #3541: Core should not add/remove content to a repository or create a repository_version without plugin input NEW Actions

History

#1 Updated by daviddavis 6 months ago

  • Description updated (diff)

#2 Updated by daviddavis 6 months ago

  • Blocks Task #4028: Prevent duplicate files in repositories added

#3 Updated by daviddavis 6 months ago

  • Sprint/Milestone set to 3.0

#4 Updated by amacdona@redhat.com 6 months ago

This was part of the intended use of the RemoveDuplicates stage. Can you specify why that doesn't work in this case?

https://pulp.plan.io/issues/3934

#5 Updated by daviddavis 6 months ago

This is just adding units to a repo (or repo version) not syncing. Here's an example from pulp_file:

http POST http://localhost:24817/pulp/api/v3/repositories/ name=foo
export REPO_HREF=$(http :24817/pulp/api/v3/repositories/ | jq -r '.results[] | select(.name == "foo") | ._href')
echo "blah1" > 1.iso
echo "blah2" > 2.iso
export ART1=$(http --form POST http://localhost:24817/pulp/api/v3/artifacts/ file@./1.iso | jq -r '._href')
export ART2=$(http --form POST http://localhost:24817/pulp/api/v3/artifacts/ file@./2.iso | jq -r '._href')
export CU1=$(http POST http://localhost:24817/pulp/api/v3/content/file/files/ relative_path=1.iso _artifact="$ART1" | jq -r '._href')
export CU2=$(http POST http://localhost:24817/pulp/api/v3/content/file/files/ relative_path=1.iso _artifact="$ART2" | jq -r '._href')
http POST ':24817'$REPO_HREF'versions/' add_content_units:="[\"$CU1\", \"$CU2\"]" 
sleep 5
http ":24817${REPO_HREF}versions/1/" 

This creates a repository with two conflicting files (ie two files with the same name).

#6 Updated by bmbouter 6 months ago

This seems similar to the story from https://pulp.plan.io/issues/3541

We want core to provide generic functionality yet only plugins can know and 'validate' the repo. I was pitching the hooking option about a year ago which has the nice default of having plugin writers take no action if they don't require extra validation and a clear mechanism they can provide if they do. https://pulp.plan.io/issues/3541?pn=1#note-3

#7 Updated by daviddavis 6 months ago

  • Blocks deleted (Task #4028: Prevent duplicate files in repositories)

#8 Updated by daviddavis 6 months ago

  • Status changed from NEW to CLOSED - DUPLICATE

#9 Updated by daviddavis 6 months ago

  • Duplicates Issue #3541: Core should not add/remove content to a repository or create a repository_version without plugin input added

Please register to edit this issue

Also available in: Atom PDF