Project

Profile

Help

Issue #8133

Same package from different repos yields error on sync

Added by adam.winberg@smhi.se 8 months ago. Updated about 1 month ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 102
Quarter:

Description

Trying to sync repos from https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-8-x86_64/ and https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7-x86_64/

gives the following error

ValueError: No declared artifact with relative path "pgdg-redhat-repo-42.0-9.noarch.rpm" for content "<Package: pgdg-redhat-repo>"

Both repos contains the same 'pgdg-redhat-repo' packages. As the packages are downloaded from one source, it then fails to download from the other.

This is on a rpm based RHEL8 installation with

python3-pulp-rpm-3.7.0-1.el8.noarch
python3-pulpcore-3.7.3-1.el8.noarch

Related issues

Related to RPM Support - Issue #7208: Error during sync : Path is duplicatedCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to RPM Support - Issue #7507: Pulp RPM 3.6.2 - ValueError: Cannot create repository version. Path is duplicatedCLOSED - DUPLICATE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to Pulp - Backport #9446: Backport #8133 "Same package from different repos yields error on sync" to 3.9.zNEW

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Copied to Pulp - Issue #9130: Backport #8133 "Same package from different repos yields error on sync" to 3.14.zCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Copied to Pulp - Backport #9227: Backport #8133 "Same package from different repos yields error on sync" to 3.7.zCLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 349f2137 View on GitHub
Added by ggainey about 2 months ago

Loosens the constraints around what can be in a new-repository-version.

There are RPM repositories "in the wild" that violate Pulp's assumptions about what is 'legal' in an incoming repository version. This commit changes some fatal errors into log-warnings, along with some heuristics around how to get a 'reasonable' repository in the face of suboptimal data.

To control activation of this behavior, RemoteArtifactSaver recognizes a ctor-keyword "fix_mismatched_remote_artifacts". This defaults to False; if provided as True, invokes the new behavior.

This change also adds handling for an exception during repo-version-failure-cleanup that could result in losing error info.

fixes #8133

History

#1 Updated by dalley 8 months ago

  • Related to Issue #7208: Error during sync : Path is duplicated added

#2 Updated by dalley 8 months ago

  • Triaged changed from No to Yes

#3 Updated by dalley 8 months ago

  • Related to Issue #7507: Pulp RPM 3.6.2 - ValueError: Cannot create repository version. Path is duplicated added

#4 Updated by adam.winberg@smhi.se 8 months ago

After a thorough cleaning - removing the offending package, deleting all repoversions where it was present and deleting orphans - I can no longer reproduce this error. I suspect there was something in the metadata from my pulp2 migration that caused this.

#5 Updated by gvde 4 months ago

I still have the same problem. It's been discussed here: https://community.theforeman.org/t/foreman-2-4-katello-4-unable-to-sync-repos/23646

Possibly related with multiple rpms having the identical checksum?

#6 Updated by trendyandy 3 months ago

I have the same issue with syncing postgresql and grafana. What can we do to bring this forward?

#7 Updated by jsherril@redhat.com 3 months ago

  • Tags Katello added

#8 Updated by martin 2 months ago

trendyandy wrote:

I have the same issue with syncing postgresql and grafana. What can we do to bring this forward?

we are impacted by this, also with Grafana repo

#9 Updated by iballou 2 months ago

Especially for Katello users in this thread, it would be good to know if Pulpcore 3.14 helps out with this issue. Katello 4.1 is now on Pulpcore 3.14 (and related plugins).

#10 Updated by gvde 2 months ago

Briefly checked: on 4.1.1 with pulpcore 3.14. grafana sync this morning:

No declared artifact with relative path "grafana-2.6.0.x86_64.rpm" for content "<Package: grafana>"

#11 Updated by gvde 2 months ago

For PostgreSQL it seems the repository changed: the repo file rpms are now at https://download.postgresql.org/pub/repos/yum/reporpms/ and not in the general repositories anymore. So there is no problem anymore.

Currently, I see it with grafana and also with eduvpn:

https://repo.letsconnect-vpn.org/2/rpm/epel-7-x86_64/

Cannot create repository version. Path is duplicated: php-fkooman-secookie-5.0.2-1.el7.noarch.rpm.

#12 Updated by ggainey 2 months ago

  • Project changed from RPM Support to Pulp

The fix for this particular failure-scenario is in artifact_stages in pulpcore. Switching to the correct project.

#13 Updated by pulpbot 2 months ago

  • Status changed from NEW to POST

#14 Updated by dalley 2 months ago

  • Assignee set to ggainey
  • Sprint/Milestone set to 3.15.0
  • Sprint set to Sprint 101

#15 Updated by dalley 2 months ago

  • Copied to Issue #9130: Backport #8133 "Same package from different repos yields error on sync" to 3.14.z added

#17 Updated by ipanova@redhat.com about 2 months ago

  • Sprint changed from Sprint 101 to Sprint 102

#18 Updated by ggainey about 2 months ago

  • Status changed from POST to MODIFIED

#19 Updated by ttereshc about 2 months ago

  • Copied to Backport #9227: Backport #8133 "Same package from different repos yields error on sync" to 3.7.z added

#20 Updated by pulpbot about 1 month ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

#21 Updated by jsherril@redhat.com 3 days ago

  • Related to Backport #9446: Backport #8133 "Same package from different repos yields error on sync" to 3.9.z added

Please register to edit this issue

Also available in: Atom PDF