Project

Profile

Help

Issue #8133

closed

Same package from different repos yields error on sync

Added by adam.winberg@smhi.se about 3 years ago. Updated over 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 102
Quarter:

Description

Trying to sync repos from https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-8-x86_64/ and https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7-x86_64/

gives the following error

ValueError: No declared artifact with relative path "pgdg-redhat-repo-42.0-9.noarch.rpm" for content "<Package: pgdg-redhat-repo>"

Both repos contains the same 'pgdg-redhat-repo' packages. As the packages are downloaded from one source, it then fails to download from the other.

This is on a rpm based RHEL8 installation with

python3-pulp-rpm-3.7.0-1.el8.noarch
python3-pulpcore-3.7.3-1.el8.noarch

Related issues

Related to RPM Support - Issue #7208: Error during sync : Path is duplicatedCLOSED - CURRENTRELEASEggaineyActions
Related to RPM Support - Issue #7507: Pulp RPM 3.6.2 - ValueError: Cannot create repository version. Path is duplicatedCLOSED - DUPLICATEActions
Related to Pulp - Backport #9446: Backport #8133 "Same package from different repos yields error on sync" to 3.9.zCLOSED - WONTFIX

Actions
Copied to Pulp - Issue #9130: Backport #8133 "Same package from different repos yields error on sync" to 3.14.zCLOSED - CURRENTRELEASEActions
Copied to Pulp - Backport #9227: Backport #8133 "Same package from different repos yields error on sync" to 3.7.zCLOSED - CURRENTRELEASEttereshc

Actions
Actions #1

Updated by dalley about 3 years ago

  • Related to Issue #7208: Error during sync : Path is duplicated added
Actions #2

Updated by dalley about 3 years ago

  • Triaged changed from No to Yes
Actions #3

Updated by dalley about 3 years ago

  • Related to Issue #7507: Pulp RPM 3.6.2 - ValueError: Cannot create repository version. Path is duplicated added
Actions #4

Updated by adam.winberg@smhi.se about 3 years ago

After a thorough cleaning - removing the offending package, deleting all repoversions where it was present and deleting orphans - I can no longer reproduce this error. I suspect there was something in the metadata from my pulp2 migration that caused this.

Actions #5

Updated by gvde almost 3 years ago

I still have the same problem. It's been discussed here: https://community.theforeman.org/t/foreman-2-4-katello-4-unable-to-sync-repos/23646

Possibly related with multiple rpms having the identical checksum?

Actions #6

Updated by trendyandy over 2 years ago

I have the same issue with syncing postgresql and grafana. What can we do to bring this forward?

Actions #7

Updated by jsherril@redhat.com over 2 years ago

  • Tags Katello added
Actions #8

Updated by martin over 2 years ago

trendyandy wrote:

I have the same issue with syncing postgresql and grafana. What can we do to bring this forward?

we are impacted by this, also with Grafana repo

Actions #9

Updated by iballou over 2 years ago

Especially for Katello users in this thread, it would be good to know if Pulpcore 3.14 helps out with this issue. Katello 4.1 is now on Pulpcore 3.14 (and related plugins).

Actions #10

Updated by gvde over 2 years ago

Briefly checked: on 4.1.1 with pulpcore 3.14. grafana sync this morning:

No declared artifact with relative path "grafana-2.6.0.x86_64.rpm" for content "<Package: grafana>"

Actions #11

Updated by gvde over 2 years ago

For PostgreSQL it seems the repository changed: the repo file rpms are now at https://download.postgresql.org/pub/repos/yum/reporpms/ and not in the general repositories anymore. So there is no problem anymore.

Currently, I see it with grafana and also with eduvpn:

https://repo.letsconnect-vpn.org/2/rpm/epel-7-x86_64/

Cannot create repository version. Path is duplicated: php-fkooman-secookie-5.0.2-1.el7.noarch.rpm.
Actions #12

Updated by ggainey over 2 years ago

  • Project changed from RPM Support to Pulp

The fix for this particular failure-scenario is in artifact_stages in pulpcore. Switching to the correct project.

Actions #13

Updated by pulpbot over 2 years ago

  • Status changed from NEW to POST
Actions #14

Updated by dalley over 2 years ago

  • Assignee set to ggainey
  • Sprint/Milestone set to 3.15.0
  • Sprint set to Sprint 101
Actions #15

Updated by dalley over 2 years ago

  • Copied to Issue #9130: Backport #8133 "Same package from different repos yields error on sync" to 3.14.z added
Actions #17

Updated by ipanova@redhat.com over 2 years ago

  • Sprint changed from Sprint 101 to Sprint 102

Added by ggainey over 2 years ago

Revision 349f2137 | View on GitHub

Loosens the constraints around what can be in a new-repository-version.

There are RPM repositories "in the wild" that violate Pulp's assumptions about what is 'legal' in an incoming repository version. This commit changes some fatal errors into log-warnings, along with some heuristics around how to get a 'reasonable' repository in the face of suboptimal data.

To control activation of this behavior, RemoteArtifactSaver recognizes a ctor-keyword "fix_mismatched_remote_artifacts". This defaults to False; if provided as True, invokes the new behavior.

This change also adds handling for an exception during repo-version-failure-cleanup that could result in losing error info.

fixes #8133

Actions #18

Updated by ggainey over 2 years ago

  • Status changed from POST to MODIFIED
Actions #19

Updated by ttereshc over 2 years ago

  • Copied to Backport #9227: Backport #8133 "Same package from different repos yields error on sync" to 3.7.z added
Actions #20

Updated by pulpbot over 2 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #21

Updated by jsherril@redhat.com over 2 years ago

  • Related to Backport #9446: Backport #8133 "Same package from different repos yields error on sync" to 3.9.z added

Also available in: Atom PDF