Project

Profile

Help

Send by e-mail

Issue #7507

Pulp RPM 3.6.2 - ValueError: Cannot create repository version. Path is duplicated

Added by abrix about 1 year ago. Updated 4 months ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Hi,

with Pulp RPM 3.3.0 issue #6303 should be fixed, but with release 3.6.2 it appears to be back.

Best regards, Alex

$ http --body --pretty=format --verify=no GET https://localhost/pulp/api/v3/status/ |jq '.versions'
[
  {
    "component": "pulpcore",
    "version": "3.6.3"
  },
  {
    "component": "pulp_rpm",
    "version": "3.6.2"
  },
        [...]
$ sudo journalctl --since=today -t rq
        [...]
Sep 15 13:56:19 pulp01 rq[27407]: pulp: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-sles-12sp5-x86_64-backports-standard remote=upstream-sles-12sp5-x86_64-backports-standard
Sep 15 13:57:48 pulp01 rq[27407]: pulp: rq.worker:ERROR: Traceback (most recent call last):
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 126, in validate_version_paths
Sep 15 13:57:48 pulp01 rq[27407]: validate_file_paths(paths)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/files.py", line 132, in validate_file_paths
Sep 15 13:57:48 pulp01 rq[27407]: raise ValueError(_("Path is duplicated: {path}").format(path=path))
Sep 15 13:57:48 pulp01 rq[27407]: ValueError: Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm
Sep 15 13:57:48 pulp01 rq[27407]: During handling of the above exception, another exception occurred:
Sep 15 13:57:48 pulp01 rq[27407]: Traceback (most recent call last):
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Sep 15 13:57:48 pulp01 rq[27407]: rv = job.perform()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 684, in perform
Sep 15 13:57:48 pulp01 rq[27407]: self._result = self._execute()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 690, in _execute
Sep 15 13:57:48 pulp01 rq[27407]: return self.func(*self.args, **self.kwargs)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 266, in synchronize
Sep 15 13:57:48 pulp01 rq[27407]: dv.create()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create
Sep 15 13:57:48 pulp01 rq[27407]: loop.run_until_complete(pipeline)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/repository.py", line 790, in __exit__
Sep 15 13:57:48 pulp01 rq[27407]: repository.finalize_new_version(self)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/models/repository.py", line 238, in finalize_new_version
Sep 15 13:57:48 pulp01 rq[27407]: validate_repo_version(new_version)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 141, in validate_repo_version
Sep 15 13:57:48 pulp01 rq[27407]: validate_version_paths(version)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 128, in validate_version_paths
Sep 15 13:57:48 pulp01 rq[27407]: raise ValueError(_("Cannot create repository version. {err}.").format(err=e))
Sep 15 13:57:48 pulp01 rq[27407]: ValueError: Cannot create repository version. Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm.
Sep 15 13:57:48 pulp01 rq[27407]: Traceback (most recent call last):
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 126, in validate_version_paths
Sep 15 13:57:48 pulp01 rq[27407]: validate_file_paths(paths)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/files.py", line 132, in validate_file_paths
Sep 15 13:57:48 pulp01 rq[27407]: raise ValueError(_("Path is duplicated: {path}").format(path=path))
Sep 15 13:57:48 pulp01 rq[27407]: ValueError: Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm
Sep 15 13:57:48 pulp01 rq[27407]: During handling of the above exception, another exception occurred:
Sep 15 13:57:48 pulp01 rq[27407]: Traceback (most recent call last):
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Sep 15 13:57:48 pulp01 rq[27407]: rv = job.perform()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 684, in perform
Sep 15 13:57:48 pulp01 rq[27407]: self._result = self._execute()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 690, in _execute
Sep 15 13:57:48 pulp01 rq[27407]: return self.func(*self.args, **self.kwargs)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 266, in synchronize
Sep 15 13:57:48 pulp01 rq[27407]: dv.create()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create
Sep 15 13:57:48 pulp01 rq[27407]: loop.run_until_complete(pipeline)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/repository.py", line 790, in __exit__
Sep 15 13:57:48 pulp01 rq[27407]: repository.finalize_new_version(self)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/models/repository.py", line 238, in finalize_new_version
Sep 15 13:57:48 pulp01 rq[27407]: validate_repo_version(new_version)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 141, in validate_repo_version
Sep 15 13:57:48 pulp01 rq[27407]: validate_version_paths(version)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 128, in validate_version_paths
Sep 15 13:57:48 pulp01 rq[27407]: raise ValueError(_("Cannot create repository version. {err}.").format(err=e))
Sep 15 13:57:48 pulp01 rq[27407]: ValueError: Cannot create repository version. Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm.
Sep 15 13:57:48 pulp01 rq[27407]: pulp: rq.worker:INFO: 27407@pulp01.example.com: 8102facb-57d4-4648-8d40-acbad546d25c
Sep 15 13:57:48 pulp01 rq[27407]: pulp: rq.worker:INFO: 27407@pulp01.example.com: Job OK (8102facb-57d4-4648-8d40-acbad546d25c)

Related issues

Related to Pulp - Issue #8133: Same package from different repos yields error on syncCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Is duplicate of RPM Support - Issue #7208: Error during sync : Path is duplicatedCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

History

#1 Updated by ttereshc about 1 year ago

Please provide a link to the repository you are trying to sync, so we can reproduce.

#2 Updated by ttereshc about 1 year ago

The root cause is likely different from #6303, even though the error looks the same. In #6303 it was a problem with modular content. Here it looks like a normal rpm, potentially duplicated within a repo.

#3 Updated by ipanova@redhat.com about 1 year ago

ttereshc wrote:

The root cause is likely different from #6303, even though the error looks the same. In #6303 it was a problem with modular content. Here it looks like a normal rpm, potentially duplicated within a repo.

I have tried searching for this package and the only place i have found it was in SP3 and there was just one rpm mentioned in the primary.xml http://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-12-SP3/standard/x86_64/

#4 Updated by abrix about 1 year ago

wrote:

ttereshc wrote:

The root cause is likely different from #6303, even though the error looks the same. In #6303 it was a problem with modular content. Here it looks like a normal rpm, potentially duplicated within a repo.

I have tried searching for this package and the only place i have found it was in SP3 and there was just one rpm mentioned in the primary.xml http://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-12-SP3/standard/x86_64/

ttereshc wrote:

Please provide a link to the repository you are trying to sync, so we can reproduce.

I tried to sync the Backports repositories from SUSE ( SLES12 SP4 and SLES12 SP5 ). It may be possible that this RPM "libColPack0-1.0.9-2.1.x86_64.rpm" is included in some other SLES repositories which are already synced from SUSE. I'll verify this and try to reproduce the error.

Anyway this error "Cannot create repository version. Path is duplicated" shouldn't be the case, because as a user I cannot controll the vendor repositories for any duplicate RPMs and Pulp should just continue with the repo sync here.

Added repos with the error:

    "error": {
        "description": "Cannot create repository version. Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm.",
        "traceback": "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 936, in perform_job\n    rv = job.perform()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 684, in perform\n    self._result = self._execute()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 690, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 266, in synchronize\n    dv.create()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 148, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/repository.py\", line 790, in __exit__\n    repository.finalize_new_version(self)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/models/repository.py\", line 238, in finalize_new_version\n    validate_repo_version(new_version)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py\", line 141, in validate_repo_version\n    validate_version_paths(version)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py\", line 128, in validate_version_paths\n    raise ValueError(_(\"Cannot create repository version. {err}.\").format(err=e))\n"
    },

    "error": {
        "description": "Cannot create repository version. Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm.",
        "traceback": "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 936, in perform_job\n    rv = job.perform()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 684, in perform\n    self._result = self._execute()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 690, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 266, in synchronize\n    dv.create()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 148, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/repository.py\", line 790, in __exit__\n    repository.finalize_new_version(self)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/models/repository.py\", line 238, in finalize_new_version\n    validate_repo_version(new_version)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py\", line 141, in validate_repo_version\n    validate_version_paths(version)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py\", line 128, in validate_version_paths\n    raise ValueError(_(\"Cannot create repository version. {err}.\").format(err=e))\n"
    },

#5 Updated by ttereshc about 1 year ago

  • Triaged changed from No to Yes

#6 Updated by abrix 11 months ago

Any update on this issue? The issue is similar to #6303 [1].

[1] https://pulp.plan.io/issues/6303

#7 Updated by dalley 10 months ago

  • Related to Issue #8133: Same package from different repos yields error on sync added

#8 Updated by dalley 10 months ago

Hello @abrix, we're hoping to get to this (and a few related issues) soon. It's a high priority, we just have a lot of work going on right now, so the earliest we can get to it is some time next month.

#9 Updated by dalley 10 months ago

  • Related to Issue #7208: Error during sync : Path is duplicated added

#10 Updated by ggainey 4 months ago

  • Status changed from NEW to CLOSED - DUPLICATE

I'm closing this as a dup of 7208, which I have just submitted a PR for. Ugly repos require, alas, ugly workarounds.

Test scripts for a variety of rel-path-weirdness scenarios can be found here : https://github.com/ggainey/pulp_startup/tree/main/rel_path

#11 Updated by ttereshc 4 months ago

  • Related to deleted (Issue #7208: Error during sync : Path is duplicated)

#12 Updated by ttereshc 4 months ago

  • Is duplicate of Issue #7208: Error during sync : Path is duplicated added

Please register to edit this issue

Send by e-mail

Also available in: Atom PDF