Migration Plugin

In Pulp 2, it was possible to add N different files with the same filename (relative path) to the same repo.
Only one is published but according to the database there are N files with the same name in one repo.
For that reason, migration plugin tries to create a repo version with all N files with the same name. It can happen only on the system which was upgraded from pulp <= 2.13.3. Here is the failure:

 pulp: rq.worker:ERROR: Traceback (most recent call last):
 File "/usr/lib/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 124, in validate_version_paths
 validate_file_paths(paths)
 File "/usr/lib/python3.6/site-packages/pulpcore/app/files.py", line 133, in validate_file_paths
 raise ValueError(_("Path is duplicated: {path}").format(path=path))
 ValueError: Path is duplicated: katello-installer-3.0.0.88.tar.gz
 During handling of the above exception, another exception occurred:
 Traceback (most recent call last):
 File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
 rv = job.perform()
 File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
 self._result = self._execute()
 File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
 return self.func(*self.args, **self.kwargs)
 File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 194, in complex_repo_migration
 create_repo_version(progress_rv, pulp2_repo, pulp3_remote)
 File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 344, in create_repo_version
 new_version.remove_content(Content.objects.filter(pk__in=to_delete))
 File "/usr/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 795, in __exit__
 repository.finalize_new_version(self)
 File "/usr/lib/python3.6/site-packages/pulp_file/app/models.py", line 79, in finalize_new_version
 validate_repo_version(new_version)
 File "/usr/lib/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 139, in validate_repo_version
 validate_version_paths(version)
 File "/usr/lib/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 126, in validate_version_paths
 raise ValueError(_("Cannot create repository version. {err}.").format(err=e))
 ValueError: Cannot create repository version. Path is duplicated: katello-installer-3.0.0.88.tar.gz.

It's a Pulp 2 bug #2773 which was fixed in 2.13.3 but we should handle this case to improve customer experience. There is no easy workaround, no easy way for a user to figure out which file is published and which needs to be removed from a Pulp 2 repo to fix the problem.

Proposed solution¶

Check pulp 2 logic and make choice in the same way if it's deterministic. That will give a user exactly the same result as they have in Pulp 2.
If it's somewhat random in Pulp 2, choose the latest to add to a repo version.

Observation¶

It seems like Pulp 2 picks the first one(the oldest one) to publish, I haven't checked the code but the system I see the error on has the older file published.