Project

Profile

Help

Issue #8410

closed

syncing deb-repos in pulp 3 fails with "The path for file... overlaps"

Added by hlinden_gon about 3 years ago. Updated almost 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version - Debian:
Platform Release:
Target Release - Debian:
OS:
CentOS 7
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Trigger the bug using the following steps:

  • install CentOS 7
  • install Katello 3.18.1 using the official instructions, no extra options
  • add "Debian Buster" as a product
  • add "Debian buster main" as a repository (see attachment)
  • start repo sync

Result:

Cannot create repository version. The path for file 'dists/buster' overlaps: dists/buster/Release, dists/buster/InRelease, dists/buster/Release.gpg, dists/buster/non-free/binary-amd64/Packages.gz, dists/buster/non-free/binary-amd64/Packages.xz, dists/buster/non-free/binary-amd64/Release, dists/buster/non-free/binary-amd64/Packages, dists/buster/main/binary-amd64/Packages.gz, dists/buster/main/binary-amd64/Packages.xz, dists/buster/main/binary-amd64/Release, dists/buster/main/binary-amd64/Packages, dists/buster/contrib/binary-amd64/Packages.gz, dists/buster/contrib/binary-amd64/Packages.xz, dists/buster/contrib/binary-amd64/Release, dists/buster/contrib/binary-amd64/Packages.

Files

repo.png (71.1 KB) repo.png hlinden_gon, 03/16/2021 10:20 AM
result.png (47.8 KB) result.png hlinden_gon, 03/16/2021 10:20 AM
address.png (141 KB) address.png hlinden_gon, 03/16/2021 04:12 PM
fm_install.sh (6.3 KB) fm_install.sh stfn, 04/01/2021 05:41 PM
sec-repo.PNG (4.29 KB) sec-repo.PNG stfn, 04/08/2021 10:38 AM
error.PNG (22.5 KB) error.PNG stfn, 04/20/2021 12:38 PM

Related issues

Related to Debian Support - Backport #8556: Backport 8410 to 2.11CLOSED - CURRENTRELEASE

Actions
Related to Debian Support - Backport #8558: Backport 8410 to 2.10CLOSED - CURRENTRELEASEquba42

Actions
Related to Debian Support - Backport #8559: Backport 8410 to 2.9CLOSED - CURRENTRELEASEquba42

Actions
Has duplicate Debian Support - Issue #8339: When I sync deb repositories I get the error: The path for file 'dists/buster' overlaps: dists/buster/non-free/binary-amd64/Packages.gz, dists/buster/non-free/binary-amd64/Packages.xz, dists/buster/non-free/binary-amd64/ReleaseCLOSED - DUPLICATEActions
Has duplicate Debian Support - Issue #8096: I18n files missingCLOSED - WONTFIXActions
Blocks Debian Support - Issue #8544: Add some translation files to at least one test fixture.CLOSED - DUPLICATEActions
Actions #1

Updated by quba42 about 3 years ago

  • Has duplicate Issue #8339: When I sync deb repositories I get the error: The path for file 'dists/buster' overlaps: dists/buster/non-free/binary-amd64/Packages.gz, dists/buster/non-free/binary-amd64/Packages.xz, dists/buster/non-free/binary-amd64/Release added
Actions #2

Updated by quba42 about 3 years ago

I have not yet observed this error during development and am unsure how to try and reproduce it.

Can you tell me what pulpcore and pulp_deb version you are using? (If not I can probably find out what is included in Katello 3.18.1).

Actions #3

Updated by hlinden_gon about 3 years ago

I am currently replicating the issue in a test environment which I can give you access to. Contact me regarding that, if it helps you.

The versions of pulp_deb and pulpcore delivered by the Katello project as of today are:

python3-pulp-deb-2.7.0-1.el7.noarch

python3-pulpcore-3.7.3-1.el7.noarch

Actions #4

Updated by hlinden_gon about 3 years ago

I have successfully replicated the issue on a fresh install of Katello 3.18. This test environment can be used for further debugging. Contact me regarding access.

Actions #5

Updated by hlinden_gon about 3 years ago

Apparently Redmine has no means for direct communication. Sorry for that, here's my mail address.

Actions #6

Updated by hlinden_gon about 3 years ago

Hi - I still have the VM with the test scenario around. Do you think that would be of help to you? In that case, I'll keep it running a bit longer.

Actions #7

Updated by quba42 about 3 years ago

Actually, now that I think about it, that could be helpful. I will be in touch by email after lunch.

Sorry, I did not react to your very helpful offer sooner. ;-)

Actions #8

Updated by stfn almost 3 years ago

Hello guys,

I was wondering are there any updates on this one?

Basically, I have the same issue.

Getting the same error on both:

foreman 2.3 with katello 3.18 foreman 2.4 with katello 4

Cannot create repository version. The path for file 'dists/buster' overlaps: dists/buster/Release, dists/buster/InRelease, dists/buster/Release.gpg, dists/buster/main/binary-amd64/Packages.gz, dists/buster/main/binary-amd64/Packages.xz, dists/buster/main/binary-amd64/Release, dists/buster/main/binary-amd64/Packages.

Actions #9

Updated by quba42 almost 3 years ago

I have not been able to reproduce this in my development environment, even when using the exact same pulp_deb and pulpcore version, as well as the exact same repository.

Either there is still something different between how Katello is using Pulp and how I set up my test scenario, or this Bug is on the Katello side (which would be odd since it looks like the error is thrown from pulpcore). My colleague who integrated pulp_deb into Katello will be back from Vacation next week, so I will try to bring this up then.

Actions #10

Updated by stfn almost 3 years ago

Thanks for the quick reply.

I hope that could help you to reproduce it on CentOS 7 or maybe I'm doing something wrong

Actions #11

Updated by quba42 almost 3 years ago

I was finally able to reproduce this in a Katello 3.18.2.1 development environment. Hopefully this will allow me to make some progress investigating this.

Actions #12

Updated by hlinden_gon almost 3 years ago

quba42 wrote:

I was finally able to reproduce this in a Katello 3.18.2.1 development environment. Hopefully this will allow me to make some progress investigating this.

Neat :) I'll leave my test environment up though until you don't need it anymore. Just drop me a line at that point.

Actions #13

Updated by quba42 almost 3 years ago

I was able to find one difference between my Pulp development environment (where this sync is successful) and my Katello development environment (where it is not): Katello defaults to using Mirror on Sync: Yes while plain Pulp uses Mirror on Sync: No.

Perhaps someone could try using Mirror on Sync: No in Katello to check if that works as a workaround for this problem until a proper fix is available?

Actions #14

Updated by quba42 almost 3 years ago

Using Mirror on Sync: No does not appear to make a difference.

Actions #15

Updated by stfn almost 3 years ago

still the same error with "Mirror on Sync: No" / Katello 4.0

Actions #16

Updated by quba42 almost 3 years ago

I am 80% sure I have found the bug. It looks like the relative_path for translation files is not being built correctly.

The real question is why this did not fail more often... I will start implementing and testing a fix tomorrow.

Actions #17

Updated by quba42 almost 3 years ago

Actions #18

Updated by stfn almost 3 years ago

Thanks for the help.

Historically, I was playing with that for a week, trying different versions, but only managed to sync the main repository once (Katello 3.18).

Another thing is that the sync on Debian 10 security-updates repo could be successfully completed but it's always empty.

URL: http://deb.debian.org/debian-security/ Releases: buster/updates Component: main Architecture: amd64

At the same time, the other two repos are working fine.

Actions #19

Updated by pulpbot almost 3 years ago

  • Status changed from NEW to POST
Actions #20

Updated by quba42 almost 3 years ago

  • Blocks Issue #8544: Add some translation files to at least one test fixture. added
Actions #21

Updated by quba42 almost 3 years ago

  • Status changed from POST to MODIFIED
Actions #22

Updated by quba42 almost 3 years ago

Actions #23

Updated by quba42 almost 3 years ago

Actions #24

Updated by quba42 almost 3 years ago

Actions #25

Updated by stfn almost 3 years ago

many thanks for the efforts

Is there a way to implement the fix on kattelo (3.18) (pulp-deb-plugins-1.10.2) yet?

I've updated the following row at synchronizing.py but that doesn't fix it on my end?

relative_path = os.path.join(os.path.dirname(release_file.relative_path), path)

Actions #26

Updated by quba42 almost 3 years ago

stfn wrote:

many thanks for the efforts

Is there a way to implement the fix on kattelo (3.18) (pulp-deb-plugins-1.10.2) yet?

I am working on getting a fixed version of pulp_deb into the repository used by Katello (for 3.18 and newer). However there is some worry that this might require newer API bindings on the Katello side, which is why this has not yet happened. Once the following PR is merged, Katello installations should have access to updated RPM packages: https://github.com/theforeman/pulpcore-packaging/pull/144

I've updated the following row at synchronizing.py but that doesn't fix it on my end?

relative_path = os.path.join(os.path.dirname(release_file.relative_path), path)

If you have manually patched this on your system (for me the file is at /usr/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py), that should fix the problem, provided the change has fully taken effect, and there is no interference from objects created by previous failed sync attempts. Things you could try to ensure this is the case:

  • Restart pulp services: systemctl restart pulp*
  • Restart Foreman/Katello services: foreman-maintain service restart
  • If that still does not fix the problem, perhaps running a Pulp orphan cleanup might help (Not sure what the command to do this is, but I believe there is a rake task for it).

I am not sure which of these steps is strictly necessary. I am pretty sure the fix itself works since I tested it on a system that reliably reproduced the error without it.

Actions #27

Updated by stfn almost 3 years ago

I've applied the fix suggested, but once you try to sync/create a repo after that, the following error occurs.

I guess Katello should also update their code.

Actions #28

Updated by quba42 almost 3 years ago

stfn wrote:

I've applied the fix suggested, but once you try to sync/create a repo after that, the following error occurs.

I guess Katello should also update their code.

Did you update to a newer pulp_deb version, or did you just manually patch the one line in /usr/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py?

Edit: Also check if all your pulp services are running properly, the error suggests they might not have started correctly. (systemctl status pulp*)

Actions #29

Updated by stfn almost 3 years ago

quba42 wrote:

Did you update to a newer pulp_deb version, or did you just manually patch the one line in /usr/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py?

I've manually patched it.

Transaction ID : 31
Begin time     : Tue Apr 20 11:38:05 2021
Package        : pulp-deb-plugins-1.10.2-1.el7.noarch
State          : Install
Size           : 232,774
Build host     : koji.katello.org
Build time     : Thu Oct 29 21:25:39 2020
Packager       : Koji
Vendor         : Koji
License        : GPLv2
URL            : https://github.com/pulp/pulp_deb
Source RPM     : pulp-deb-1.10.2-1.el7.src.rpm
Commit Time    : Wed Mar  4 13:00:00 2020
Committer      : Evgeni Golov - 1.10.1-1
Reason         : user
Command Line   : -d 0 -e 0 -y install pulp-deb-plugins
From repo      : pulp
Installed by   : root <root>

where can I get the patched version?

Edit: Also check if all your pulp services are running properly, the error suggests they might not have started correctly. (systemctl status pulp*)

after reboot everything is running, and on a later stage some of the workeras start failing.. for example

● pulpcore-worker@3.service - Pulp RQ Worker
   Loaded: loaded (/etc/systemd/system/pulpcore-worker@.service; enabled; vendor preset: disabled)
   Active: activating (auto-restart) (Result: exit-code) since Tue 2021-04-20 15:12:23 CEST; 963ms ago
  Process: 12061 ExecStart=/usr/libexec/pulpcore/rq worker -w pulpcore.tasking.worker.PulpWorker -c pulpcore.rqconfig --disable-job-desc-logging (code=exited, status=1/FAILURE)
 Main PID: 12061 (code=exited, status=1/FAILURE)
   CGroup: /system.slice/system-pulpcore\x2dworker.slice/pulpcore-worker@3.service

Apr 20 15:12:23 localhost systemd[1]: Unit pulpcore-worker@3.service entered failed state.
Apr 20 15:12:23 localhost systemd[1]: pulpcore-worker@3.service failed.
Actions #30

Updated by stfn almost 3 years ago

I mean I've tried to get the latest source from here and follow the instructions, but these assume that you have installed the pulpcore from source as well, which is not my case, so it doesn't work this way, I guess just waiting for an rpm package would be the easiest option.

https://github.com/pulp/pulp_deb https://docs.pulpproject.org/pulp_deb/installation.html

Actions #31

Updated by quba42 almost 3 years ago

  • Sprint/Milestone set to 2.12.0
Actions #32

Updated by pulpbot almost 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF