Project

Profile

Help

Send by e-mail

Issue #7507

Pulp RPM 3.6.2 - ValueError: Cannot create repository version. Path is duplicated

Added by abrix 7 months ago. Updated 3 months ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Hi,

with Pulp RPM 3.3.0 issue #6303 should be fixed, but with release 3.6.2 it appears to be back.

Best regards, Alex

$ http --body --pretty=format --verify=no GET https://localhost/pulp/api/v3/status/ |jq '.versions'
[
  {
    "component": "pulpcore",
    "version": "3.6.3"
  },
  {
    "component": "pulp_rpm",
    "version": "3.6.2"
  },
        [...]
$ sudo journalctl --since=today -t rq
        [...]
Sep 15 13:56:19 pulp01 rq[27407]: pulp: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-sles-12sp5-x86_64-backports-standard remote=upstream-sles-12sp5-x86_64-backports-standard
Sep 15 13:57:48 pulp01 rq[27407]: pulp: rq.worker:ERROR: Traceback (most recent call last):
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 126, in validate_version_paths
Sep 15 13:57:48 pulp01 rq[27407]: validate_file_paths(paths)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/files.py", line 132, in validate_file_paths
Sep 15 13:57:48 pulp01 rq[27407]: raise ValueError(_("Path is duplicated: {path}").format(path=path))
Sep 15 13:57:48 pulp01 rq[27407]: ValueError: Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm
Sep 15 13:57:48 pulp01 rq[27407]: During handling of the above exception, another exception occurred:
Sep 15 13:57:48 pulp01 rq[27407]: Traceback (most recent call last):
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Sep 15 13:57:48 pulp01 rq[27407]: rv = job.perform()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 684, in perform
Sep 15 13:57:48 pulp01 rq[27407]: self._result = self._execute()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 690, in _execute
Sep 15 13:57:48 pulp01 rq[27407]: return self.func(*self.args, **self.kwargs)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 266, in synchronize
Sep 15 13:57:48 pulp01 rq[27407]: dv.create()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create
Sep 15 13:57:48 pulp01 rq[27407]: loop.run_until_complete(pipeline)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/repository.py", line 790, in __exit__
Sep 15 13:57:48 pulp01 rq[27407]: repository.finalize_new_version(self)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/models/repository.py", line 238, in finalize_new_version
Sep 15 13:57:48 pulp01 rq[27407]: validate_repo_version(new_version)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 141, in validate_repo_version
Sep 15 13:57:48 pulp01 rq[27407]: validate_version_paths(version)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 128, in validate_version_paths
Sep 15 13:57:48 pulp01 rq[27407]: raise ValueError(_("Cannot create repository version. {err}.").format(err=e))
Sep 15 13:57:48 pulp01 rq[27407]: ValueError: Cannot create repository version. Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm.
Sep 15 13:57:48 pulp01 rq[27407]: Traceback (most recent call last):
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 126, in validate_version_paths
Sep 15 13:57:48 pulp01 rq[27407]: validate_file_paths(paths)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/files.py", line 132, in validate_file_paths
Sep 15 13:57:48 pulp01 rq[27407]: raise ValueError(_("Path is duplicated: {path}").format(path=path))
Sep 15 13:57:48 pulp01 rq[27407]: ValueError: Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm
Sep 15 13:57:48 pulp01 rq[27407]: During handling of the above exception, another exception occurred:
Sep 15 13:57:48 pulp01 rq[27407]: Traceback (most recent call last):
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Sep 15 13:57:48 pulp01 rq[27407]: rv = job.perform()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 684, in perform
Sep 15 13:57:48 pulp01 rq[27407]: self._result = self._execute()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 690, in _execute
Sep 15 13:57:48 pulp01 rq[27407]: return self.func(*self.args, **self.kwargs)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 266, in synchronize
Sep 15 13:57:48 pulp01 rq[27407]: dv.create()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create
Sep 15 13:57:48 pulp01 rq[27407]: loop.run_until_complete(pipeline)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/repository.py", line 790, in __exit__
Sep 15 13:57:48 pulp01 rq[27407]: repository.finalize_new_version(self)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/models/repository.py", line 238, in finalize_new_version
Sep 15 13:57:48 pulp01 rq[27407]: validate_repo_version(new_version)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 141, in validate_repo_version
Sep 15 13:57:48 pulp01 rq[27407]: validate_version_paths(version)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 128, in validate_version_paths
Sep 15 13:57:48 pulp01 rq[27407]: raise ValueError(_("Cannot create repository version. {err}.").format(err=e))
Sep 15 13:57:48 pulp01 rq[27407]: ValueError: Cannot create repository version. Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm.
Sep 15 13:57:48 pulp01 rq[27407]: pulp: rq.worker:INFO: 27407@pulp01.example.com: 8102facb-57d4-4648-8d40-acbad546d25c
Sep 15 13:57:48 pulp01 rq[27407]: pulp: rq.worker:INFO: 27407@pulp01.example.com: Job OK (8102facb-57d4-4648-8d40-acbad546d25c)

Related issues

Related to RPM Support - Issue #8133: Same package from different repos yields error on syncNEW<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to RPM Support - Issue #7208: Error during sync : Path is duplicatedNEW<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

History

#1 Updated by ttereshc 7 months ago

Please provide a link to the repository you are trying to sync, so we can reproduce.

#2 Updated by ttereshc 7 months ago

The root cause is likely different from #6303, even though the error looks the same. In #6303 it was a problem with modular content. Here it looks like a normal rpm, potentially duplicated within a repo.

#3 Updated by ipanova@redhat.com 7 months ago

ttereshc wrote:

The root cause is likely different from #6303, even though the error looks the same. In #6303 it was a problem with modular content. Here it looks like a normal rpm, potentially duplicated within a repo.

I have tried searching for this package and the only place i have found it was in SP3 and there was just one rpm mentioned in the primary.xml http://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-12-SP3/standard/x86_64/

#4 Updated by abrix 7 months ago

wrote:

ttereshc wrote:

The root cause is likely different from #6303, even though the error looks the same. In #6303 it was a problem with modular content. Here it looks like a normal rpm, potentially duplicated within a repo.

I have tried searching for this package and the only place i have found it was in SP3 and there was just one rpm mentioned in the primary.xml http://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-12-SP3/standard/x86_64/

ttereshc wrote:

Please provide a link to the repository you are trying to sync, so we can reproduce.

I tried to sync the Backports repositories from SUSE ( SLES12 SP4 and SLES12 SP5 ). It may be possible that this RPM "libColPack0-1.0.9-2.1.x86_64.rpm" is included in some other SLES repositories which are already synced from SUSE. I'll verify this and try to reproduce the error.

Anyway this error "Cannot create repository version. Path is duplicated" shouldn't be the case, because as a user I cannot controll the vendor repositories for any duplicate RPMs and Pulp should just continue with the repo sync here.

Added repos with the error:

    "error": {
        "description": "Cannot create repository version. Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm.",
        "traceback": "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 936, in perform_job\n    rv = job.perform()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 684, in perform\n    self._result = self._execute()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 690, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 266, in synchronize\n    dv.create()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 148, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/repository.py\", line 790, in __exit__\n    repository.finalize_new_version(self)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/models/repository.py\", line 238, in finalize_new_version\n    validate_repo_version(new_version)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py\", line 141, in validate_repo_version\n    validate_version_paths(version)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py\", line 128, in validate_version_paths\n    raise ValueError(_(\"Cannot create repository version. {err}.\").format(err=e))\n"
    },

    "error": {
        "description": "Cannot create repository version. Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm.",
        "traceback": "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 936, in perform_job\n    rv = job.perform()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 684, in perform\n    self._result = self._execute()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 690, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 266, in synchronize\n    dv.create()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 148, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/repository.py\", line 790, in __exit__\n    repository.finalize_new_version(self)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/models/repository.py\", line 238, in finalize_new_version\n    validate_repo_version(new_version)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py\", line 141, in validate_repo_version\n    validate_version_paths(version)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py\", line 128, in validate_version_paths\n    raise ValueError(_(\"Cannot create repository version. {err}.\").format(err=e))\n"
    },

#5 Updated by ttereshc 7 months ago

  • Triaged changed from No to Yes

#6 Updated by abrix 3 months ago

Any update on this issue? The issue is similar to #6303 [1].

[1] https://pulp.plan.io/issues/6303

#7 Updated by dalley 3 months ago

  • Related to Issue #8133: Same package from different repos yields error on sync added

#8 Updated by dalley 3 months ago

Hello @abrix, we're hoping to get to this (and a few related issues) soon. It's a high priority, we just have a lot of work going on right now, so the earliest we can get to it is some time next month.

#9 Updated by dalley about 2 months ago

  • Related to Issue #7208: Error during sync : Path is duplicated added

Please register to edit this issue

Send by e-mail

Also available in: Atom PDF