Issue #7208
closedError during sync : Path is duplicated
Description
Sync repo http://ftp.cs.stanford.edu/centos/7/opstools/x86_64/ and observe traceback
$ http GET $BASE_ADDR/pulp/api/v3/tasks/91ae8e5d-bb4f-4151-a5c3-d24a3c04d81e/
HTTP/1.1 200 OK
Allow: GET, PATCH, DELETE, HEAD, OPTIONS
Connection: close
Content-Length: 2558
Content-Type: application/json
Date: Fri, 24 Jul 2020 10:42:54 GMT
Server: gunicorn/20.0.4
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"child_tasks": [],
"created_resources": [],
"error": {
"description": "Cannot create repository version. Path is duplicated: rubygem-elasticsearch-doc-1.0.8-1.el7.noarch.rpm.",
"traceback": " File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/worker.py\", line 886, in perform_job\n rv = job.perform()\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 664, in perform\n self._result = self._execute()\n File \"/usr/local/lib/pulp/lib64/python3.6/site-packages/rq/job.py\", line 670, in _execute\n return self.func(*self.args, **self.kwargs)\n File \"/home/vagrant/devel/pulp_rpm/pulp_rpm/app/tasks/synchronizing.py\", line 206, in synchronize\n dv.create()\n File \"/home/vagrant/devel/pulpcore/pulpcore/plugin/stages/declarative_version.py\", line 148, in create\n loop.run_until_complete(pipeline)\n File \"/home/vagrant/devel/pulpcore/pulpcore/app/models/repository.py\", line 775, in __exit__\n repository.finalize_new_version(self)\n File \"/home/vagrant/devel/pulp_rpm/pulp_rpm/app/models/repository.py\", line 145, in finalize_new_version\n validate_repo_version(new_version)\n File \"/home/vagrant/devel/pulpcore/pulpcore/plugin/repo_version_utils.py\", line 141, in validate_repo_version\n validate_version_paths(version)\n File \"/home/vagrant/devel/pulpcore/pulpcore/plugin/repo_version_utils.py\", line 128, in validate_version_paths\n raise ValueError(_(\"Cannot create repository version. {err}.\").format(err=e))\n"
},
"finished_at": "2020-07-24T10:40:27.431030Z",
"name": "pulp_rpm.app.tasks.synchronizing.synchronize",
"parent_task": null,
"progress_reports": [
{
"code": "downloading.metadata",
"done": 4,
"message": "Downloading Metadata Files",
"state": "completed",
"suffix": null,
"total": null
},
{
"code": "downloading.artifacts",
"done": 0,
"message": "Downloading Artifacts",
"state": "completed",
"suffix": null,
"total": null
},
{
"code": "associating.content",
"done": 1069,
"message": "Associating Content",
"state": "completed",
"suffix": null,
"total": null
},
{
"code": "parsing.packages",
"done": 1069,
"message": "Parsed Packages",
"state": "completed",
"suffix": null,
"total": 1069
}
],
"pulp_created": "2020-07-24T10:40:14.425149Z",
"pulp_href": "/pulp/api/v3/tasks/91ae8e5d-bb4f-4151-a5c3-d24a3c04d81e/",
"reserved_resources_record": [
"/pulp/api/v3/remotes/rpm/rpm/57f994cf-4caf-4c37-b631-0eefdab2b8a1/",
"/pulp/api/v3/repositories/rpm/rpm/9da9008b-937b-468a-bbaa-f3786b4b7b15/"
],
"started_at": "2020-07-24T10:40:14.597862Z",
"state": "failed",
"task_group": null,
"worker": "/pulp/api/v3/workers/9ab80819-e321-4187-83ef-4d8adafd3d80/"
}
(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 _scripts]$
Related issues
Updated by ttereshc over 4 years ago
There are 2 of them indeed, different by location_href and checksum :/
<package type="rpm">
<name>rubygem-elasticsearch-doc</name>
<arch>noarch</arch>
<version epoch="0" ver="1.0.8" rel="1.el7"/>
<checksum type="sha256" pkgid="YES">beb1c816b3df53b53e544865eb4a823989522627ac905db6c5061b45bc2f4de0</checksum>
<summary>Documentation for rubygem-elasticsearch</summary>
<description>Documentation for rubygem-elasticsearch.</description>
<packager>CBS <cbs@centos.org></packager>
<url>http://github.com/elasticsearch/elasticsearch-ruby</url>
<time file="1493661231" build="1460639359"/>
<size package="72024" installed="177000" archive="187056"/>
<location href="fluentd/rubygem-elasticsearch-doc-1.0.8-1.el7.noarch.rpm"/>
...
<package type="rpm">
<name>rubygem-elasticsearch-doc</name>
<arch>noarch</arch>
<version epoch="0" ver="1.0.8" rel="1.el7"/>
<checksum type="sha256" pkgid="YES">5be0c906ff3c21e2755c1b60da5011cb771147dfa27e817f12a7488803e71d0c</checksum>
<summary>Documentation for rubygem-elasticsearch</summary>
<description>Documentation for rubygem-elasticsearch.</description>
<packager>CBS <cbs@centos.org></packager>
<url>http://github.com/elasticsearch/elasticsearch-ruby</url>
<time file="1497443660" build="1460639359"/>
<size package="72024" installed="177000" archive="187056"/>
<location href="logging/rubygem-elasticsearch-doc-1.0.8-1.el7.noarch.rpm"/>
Updated by ttereshc over 4 years ago
as for dnf behaviour, so far I got " ttereshc: When choices are identical, then solver could pick random option."
Updated by ggainey almost 4 years ago
- Has duplicate Issue #8043: ValueError: Path is duplicated when syncing http://mirror.centos.org/centos/7/opstools/x86_64/ added
Updated by dalley almost 4 years ago
- Related to Issue #8133: Same package from different repos yields error on sync added
Updated by dalley almost 4 years ago
- Related to Issue #7507: Pulp RPM 3.6.2 - ValueError: Cannot create repository version. Path is duplicated added
Updated by adam.winberg@smhi.se almost 4 years ago
Is there any workaround to this? We have repos we cant sync because of this. When trying to remove the repos and re-add them we run into #8351. So we are stuck between bugs and it's becoming a real problem for us.
Updated by wibbit over 3 years ago
Morning all.
I'm also seeing this issue, so would be keen to see a resolution.
I'm hitting it on
{'ca_cert': None, 'client_cert': None, 'connect_timeout': None, 'download_concurrency': 10, 'headers': None, 'name': 'upstream_rhel7_influxdb', 'policy': 'immediate', 'proxy_url': None, 'pulp_created': datetime.datetime(2021, 5, 10, 19, 4, 10, 745798, tzinfo=tzutc()), 'pulp_href': '/pulp/api/v3/remotes/rpm/rpm/47e8b4af-2ff4-4b89-945a-51b5201c424c/', 'pulp_labels': {}, 'pulp_last_updated': datetime.datetime(2021, 7, 9, 9, 8, 49, 768788, tzinfo=tzutc()), 'rate_limit': None, 'sles_auth_token': None, 'sock_connect_timeout': None, 'sock_read_timeout': None, 'tls_validation': True, 'total_timeout': None, 'url': 'https://repos.influxdata.com/rhel/7Server/x86_64/stable/'}
Any thing I can do to help let me know.
Updated by pulpbot over 3 years ago
- Status changed from NEW to POST
Updated by ttereshc over 3 years ago
- Assignee set to ggainey
- Sprint set to Sprint 101
Updated by ttereshc over 3 years ago
- Related to deleted (Issue #7507: Pulp RPM 3.6.2 - ValueError: Cannot create repository version. Path is duplicated)
Updated by ttereshc over 3 years ago
- Has duplicate Issue #7507: Pulp RPM 3.6.2 - ValueError: Cannot create repository version. Path is duplicated added
Added by ggainey over 3 years ago
Updated by ggainey over 3 years ago
- Status changed from POST to MODIFIED
Applied in changeset 0152ff4204a1d0a42360bbc9ec8365131cfa6157.
Updated by dalley over 3 years ago
- Copied to Backport #9192: Backport #7208 "Error during sync : Path is duplicated" to 3.14.z added
Updated by ttereshc over 3 years ago
- Copied to Backport #9285: Backport 'Error during sync : Path is duplicated' to 3.11.z added
Updated by pulpbot over 3 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Make "is this version valid" checks more lenient.
Handles the case of "same name, multiple locations, diff content".
fixes #7208