Project

Profile

Help

Issue #8582

ISO content migration fails with ValueError: Cannot create repository version. Path is duplicated: <filename>

Added by ttereshc 9 months ago. Updated 9 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 96
Quarter:

Description

In Pulp 2, it was possible to add N different files with the same filename (relative path) to the same repo.
Only one is published but according to the database there are N files with the same name in one repo.
For that reason, migration plugin tries to create a repo version with all N files with the same name. It can happen only on the system which was upgraded from pulp <= 2.13.3. Here is the failure:

 pulp: rq.worker:ERROR: Traceback (most recent call last):
 File "/usr/lib/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 124, in validate_version_paths
 validate_file_paths(paths)
 File "/usr/lib/python3.6/site-packages/pulpcore/app/files.py", line 133, in validate_file_paths
 raise ValueError(_("Path is duplicated: {path}").format(path=path))
 ValueError: Path is duplicated: katello-installer-3.0.0.88.tar.gz
 During handling of the above exception, another exception occurred:
 Traceback (most recent call last):
 File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
 rv = job.perform()
 File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
 self._result = self._execute()
 File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
 return self.func(*self.args, **self.kwargs)
 File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 194, in complex_repo_migration
 create_repo_version(progress_rv, pulp2_repo, pulp3_remote)
 File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 344, in create_repo_version
 new_version.remove_content(Content.objects.filter(pk__in=to_delete))
 File "/usr/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 795, in __exit__
 repository.finalize_new_version(self)
 File "/usr/lib/python3.6/site-packages/pulp_file/app/models.py", line 79, in finalize_new_version
 validate_repo_version(new_version)
 File "/usr/lib/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 139, in validate_repo_version
 validate_version_paths(version)
 File "/usr/lib/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 126, in validate_version_paths
 raise ValueError(_("Cannot create repository version. {err}.").format(err=e))
 ValueError: Cannot create repository version. Path is duplicated: katello-installer-3.0.0.88.tar.gz.

It's a Pulp 2 bug #2773 which was fixed in 2.13.3 but we should handle this case to improve customer experience. There is no easy workaround, no easy way for a user to figure out which file is published and which needs to be removed from a Pulp 2 repo to fix the problem.

Proposed solution

Check pulp 2 logic and make choice in the same way if it's deterministic. That will give a user exactly the same result as they have in Pulp 2.
If it's somewhat random in Pulp 2, choose the latest to add to a repo version.

Observation

It seems like Pulp 2 picks the first one(the oldest one) to publish, I haven't checked the code but the system I see the error on has the older file published.

Associated revisions

Revision 707518a1 View on GitHub
Added by ttereshc 9 months ago

Handle overlaping paths which come from pulp2 repos

closes #8582 https://pulp.plan.io/issues/8582

Revision 707518a1 View on GitHub
Added by ttereshc 9 months ago

Handle overlaping paths which come from pulp2 repos

closes #8582 https://pulp.plan.io/issues/8582

Revision 707518a1 View on GitHub
Added by ttereshc 9 months ago

Handle overlaping paths which come from pulp2 repos

closes #8582 https://pulp.plan.io/issues/8582

Revision 4c0bee14 View on GitHub
Added by ttereshc 9 months ago

Handle overlaping paths which come from pulp2 repos

closes #8582 https://pulp.plan.io/issues/8582

(cherry picked from commit 707518a1dac56ba4b5748ed69b934185c52e84a1)

Revision 4c0bee14 View on GitHub
Added by ttereshc 9 months ago

Handle overlaping paths which come from pulp2 repos

closes #8582 https://pulp.plan.io/issues/8582

(cherry picked from commit 707518a1dac56ba4b5748ed69b934185c52e84a1)

Revision 4c0bee14 View on GitHub
Added by ttereshc 9 months ago

Handle overlaping paths which come from pulp2 repos

closes #8582 https://pulp.plan.io/issues/8582

(cherry picked from commit 707518a1dac56ba4b5748ed69b934185c52e84a1)

History

#1 Updated by ttereshc 9 months ago

  • Subject changed from migration fails with ValueError: Cannot create repository version. Path is duplicated: <filename> to ISO content migration fails with ValueError: Cannot create repository version. Path is duplicated: <filename>

#2 Updated by ttereshc 9 months ago

  • Tags Katello added

#3 Updated by ttereshc 9 months ago

  • Description updated (diff)

#4 Updated by ttereshc 9 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc

#5 Updated by ttereshc 9 months ago

  • Sprint set to Sprint 95

#6 Updated by rchan 9 months ago

  • Sprint changed from Sprint 95 to Sprint 96

#7 Updated by ttereshc 9 months ago

  • Status changed from ASSIGNED to POST

#8 Updated by ttereshc 9 months ago

  • Status changed from POST to MODIFIED

#9 Updated by ttereshc 9 months ago

  • Sprint/Milestone set to 0.11.1

#10 Updated by pulpbot 9 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF