Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #7507

closed

Pulp RPM 3.6.2 - ValueError: Cannot create repository version. Path is duplicated

Added by abrix about 2 years ago. Updated over 1 year ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Hi,

with Pulp RPM 3.3.0 issue #6303 should be fixed, but with release 3.6.2 it appears to be back.

Best regards, Alex

$ http --body --pretty=format --verify=no GET https://localhost/pulp/api/v3/status/ |jq '.versions'
[
  {
    "component": "pulpcore",
    "version": "3.6.3"
  },
  {
    "component": "pulp_rpm",
    "version": "3.6.2"
  },
        [...]
$ sudo journalctl --since=today -t rq
        [...]
Sep 15 13:56:19 pulp01 rq[27407]: pulp: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-sles-12sp5-x86_64-backports-standard remote=upstream-sles-12sp5-x86_64-backports-standard
Sep 15 13:57:48 pulp01 rq[27407]: pulp: rq.worker:ERROR: Traceback (most recent call last):
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 126, in validate_version_paths
Sep 15 13:57:48 pulp01 rq[27407]: validate_file_paths(paths)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/files.py", line 132, in validate_file_paths
Sep 15 13:57:48 pulp01 rq[27407]: raise ValueError(_("Path is duplicated: {path}").format(path=path))
Sep 15 13:57:48 pulp01 rq[27407]: ValueError: Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm
Sep 15 13:57:48 pulp01 rq[27407]: During handling of the above exception, another exception occurred:
Sep 15 13:57:48 pulp01 rq[27407]: Traceback (most recent call last):
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Sep 15 13:57:48 pulp01 rq[27407]: rv = job.perform()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 684, in perform
Sep 15 13:57:48 pulp01 rq[27407]: self._result = self._execute()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 690, in _execute
Sep 15 13:57:48 pulp01 rq[27407]: return self.func(*self.args, **self.kwargs)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 266, in synchronize
Sep 15 13:57:48 pulp01 rq[27407]: dv.create()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create
Sep 15 13:57:48 pulp01 rq[27407]: loop.run_until_complete(pipeline)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/repository.py", line 790, in __exit__
Sep 15 13:57:48 pulp01 rq[27407]: repository.finalize_new_version(self)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/models/repository.py", line 238, in finalize_new_version
Sep 15 13:57:48 pulp01 rq[27407]: validate_repo_version(new_version)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 141, in validate_repo_version
Sep 15 13:57:48 pulp01 rq[27407]: validate_version_paths(version)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 128, in validate_version_paths
Sep 15 13:57:48 pulp01 rq[27407]: raise ValueError(_("Cannot create repository version. {err}.").format(err=e))
Sep 15 13:57:48 pulp01 rq[27407]: ValueError: Cannot create repository version. Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm.
Sep 15 13:57:48 pulp01 rq[27407]: Traceback (most recent call last):
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 126, in validate_version_paths
Sep 15 13:57:48 pulp01 rq[27407]: validate_file_paths(paths)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/files.py", line 132, in validate_file_paths
Sep 15 13:57:48 pulp01 rq[27407]: raise ValueError(_("Path is duplicated: {path}").format(path=path))
Sep 15 13:57:48 pulp01 rq[27407]: ValueError: Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm
Sep 15 13:57:48 pulp01 rq[27407]: During handling of the above exception, another exception occurred:
Sep 15 13:57:48 pulp01 rq[27407]: Traceback (most recent call last):
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Sep 15 13:57:48 pulp01 rq[27407]: rv = job.perform()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 684, in perform
Sep 15 13:57:48 pulp01 rq[27407]: self._result = self._execute()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py", line 690, in _execute
Sep 15 13:57:48 pulp01 rq[27407]: return self.func(*self.args, **self.kwargs)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 266, in synchronize
Sep 15 13:57:48 pulp01 rq[27407]: dv.create()
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create
Sep 15 13:57:48 pulp01 rq[27407]: loop.run_until_complete(pipeline)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/repository.py", line 790, in __exit__
Sep 15 13:57:48 pulp01 rq[27407]: repository.finalize_new_version(self)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/models/repository.py", line 238, in finalize_new_version
Sep 15 13:57:48 pulp01 rq[27407]: validate_repo_version(new_version)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 141, in validate_repo_version
Sep 15 13:57:48 pulp01 rq[27407]: validate_version_paths(version)
Sep 15 13:57:48 pulp01 rq[27407]: File "/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 128, in validate_version_paths
Sep 15 13:57:48 pulp01 rq[27407]: raise ValueError(_("Cannot create repository version. {err}.").format(err=e))
Sep 15 13:57:48 pulp01 rq[27407]: ValueError: Cannot create repository version. Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm.
Sep 15 13:57:48 pulp01 rq[27407]: pulp: rq.worker:INFO: 27407@pulp01.example.com: 8102facb-57d4-4648-8d40-acbad546d25c
Sep 15 13:57:48 pulp01 rq[27407]: pulp: rq.worker:INFO: 27407@pulp01.example.com: Job OK (8102facb-57d4-4648-8d40-acbad546d25c)

Related issues

Related to Pulp - Issue #8133: Same package from different repos yields error on syncCLOSED - CURRENTRELEASEggaineyActions
Is duplicate of RPM Support - Issue #7208: Error during sync : Path is duplicatedCLOSED - CURRENTRELEASEggaineyActions
Actions
Copy link
 #1

Updated by ttereshc about 2 years ago

Please provide a link to the repository you are trying to sync, so we can reproduce.

Actions
Copy link
 #2

Updated by ttereshc about 2 years ago

The root cause is likely different from #6303, even though the error looks the same. In #6303 it was a problem with modular content. Here it looks like a normal rpm, potentially duplicated within a repo.

Actions
Copy link
 #3

Updated by ipanova@redhat.com about 2 years ago

ttereshc wrote:

The root cause is likely different from #6303, even though the error looks the same. In #6303 it was a problem with modular content. Here it looks like a normal rpm, potentially duplicated within a repo.

I have tried searching for this package and the only place i have found it was in SP3 and there was just one rpm mentioned in the primary.xml http://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-12-SP3/standard/x86_64/

Actions
Copy link
 #4

Updated by abrix about 2 years ago

wrote:

ttereshc wrote:

The root cause is likely different from #6303, even though the error looks the same. In #6303 it was a problem with modular content. Here it looks like a normal rpm, potentially duplicated within a repo.

I have tried searching for this package and the only place i have found it was in SP3 and there was just one rpm mentioned in the primary.xml http://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-12-SP3/standard/x86_64/

ttereshc wrote:

Please provide a link to the repository you are trying to sync, so we can reproduce.

I tried to sync the Backports repositories from SUSE ( SLES12 SP4 and SLES12 SP5 ). It may be possible that this RPM "libColPack0-1.0.9-2.1.x86_64.rpm" is included in some other SLES repositories which are already synced from SUSE. I'll verify this and try to reproduce the error.

Anyway this error "Cannot create repository version. Path is duplicated" shouldn't be the case, because as a user I cannot controll the vendor repositories for any duplicate RPMs and Pulp should just continue with the repo sync here.

Added repos with the error:

    "error": {
        "description": "Cannot create repository version. Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm.",
        "traceback": "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 936, in perform_job\n    rv = job.perform()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 684, in perform\n    self._result = self._execute()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 690, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 266, in synchronize\n    dv.create()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 148, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/repository.py\", line 790, in __exit__\n    repository.finalize_new_version(self)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/models/repository.py\", line 238, in finalize_new_version\n    validate_repo_version(new_version)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py\", line 141, in validate_repo_version\n    validate_version_paths(version)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py\", line 128, in validate_version_paths\n    raise ValueError(_(\"Cannot create repository version. {err}.\").format(err=e))\n"
    },

    "error": {
        "description": "Cannot create repository version. Path is duplicated: libColPack0-1.0.9-2.1.x86_64.rpm.",
        "traceback": "  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 936, in perform_job\n    rv = job.perform()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 684, in perform\n    self._result = self._execute()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 690, in _execute\n    return self.func(*self.args, **self.kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 266, in synchronize\n    dv.create()\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 148, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/app/models/repository.py\", line 790, in __exit__\n    repository.finalize_new_version(self)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulp_rpm/app/models/repository.py\", line 238, in finalize_new_version\n    validate_repo_version(new_version)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py\", line 141, in validate_repo_version\n    validate_version_paths(version)\n  File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py\", line 128, in validate_version_paths\n    raise ValueError(_(\"Cannot create repository version. {err}.\").format(err=e))\n"
    },
Actions
Copy link
 #5

Updated by ttereshc about 2 years ago

  • Triaged changed from No to Yes
Actions
Copy link
 #6

Updated by abrix almost 2 years ago

Any update on this issue? The issue is similar to #6303 [1].

[1] https://pulp.plan.io/issues/6303

Actions
Copy link
 #7

Updated by dalley almost 2 years ago

  • Related to Issue #8133: Same package from different repos yields error on sync added
Actions
Copy link
 #8

Updated by dalley almost 2 years ago

Hello @abrix, we're hoping to get to this (and a few related issues) soon. It's a high priority, we just have a lot of work going on right now, so the earliest we can get to it is some time next month.

Actions
Copy link
 #9

Updated by dalley almost 2 years ago

  • Related to Issue #7208: Error during sync : Path is duplicated added
Actions
Copy link
 #10

Updated by ggainey over 1 year ago

  • Status changed from NEW to CLOSED - DUPLICATE

I'm closing this as a dup of 7208, which I have just submitted a PR for. Ugly repos require, alas, ugly workarounds.

Test scripts for a variety of rel-path-weirdness scenarios can be found here : https://github.com/ggainey/pulp_startup/tree/main/rel_path

Actions
Copy link
 #11

Updated by ttereshc over 1 year ago

  • Related to deleted (Issue #7208: Error during sync : Path is duplicated)
Actions
Copy link
 #12

Updated by ttereshc over 1 year ago

  • Is duplicate of Issue #7208: Error during sync : Path is duplicated added
Actions
Send by e-mail Copy link

Also available in: Atom PDF