Debian Support

I have not yet observed this error during development and am unsure how to try and reproduce it.

Can you tell me what pulpcore and pulp_deb version you are using? (If not I can probably find out what is included in Katello 3.18.1).

I am currently replicating the issue in a test environment which I can give you access to. Contact me regarding that, if it helps you.

The versions of pulp_deb and pulpcore delivered by the Katello project as of today are:

python3-pulp-deb-2.7.0-1.el7.noarch

python3-pulpcore-3.7.3-1.el7.noarch

I have successfully replicated the issue on a fresh install of Katello 3.18. This test environment can be used for further debugging. Contact me regarding access.

Hi - I still have the VM with the test scenario around. Do you think that would be of help to you? In that case, I'll keep it running a bit longer.

Actually, now that I think about it, that could be helpful. I will be in touch by email after lunch.

Sorry, I did not react to your very helpful offer sooner. ;-)

Hello guys,

I was wondering are there any updates on this one?

Basically, I have the same issue.

Getting the same error on both:

foreman 2.3 with katello 3.18 foreman 2.4 with katello 4

Cannot create repository version. The path for file 'dists/buster' overlaps: dists/buster/Release, dists/buster/InRelease, dists/buster/Release.gpg, dists/buster/main/binary-amd64/Packages.gz, dists/buster/main/binary-amd64/Packages.xz, dists/buster/main/binary-amd64/Release, dists/buster/main/binary-amd64/Packages.

I have not been able to reproduce this in my development environment, even when using the exact same pulp_deb and pulpcore version, as well as the exact same repository.

Either there is still something different between how Katello is using Pulp and how I set up my test scenario, or this Bug is on the Katello side (which would be odd since it looks like the error is thrown from pulpcore). My colleague who integrated pulp_deb into Katello will be back from Vacation next week, so I will try to bring this up then.

I was finally able to reproduce this in a Katello 3.18.2.1 development environment. Hopefully this will allow me to make some progress investigating this.

quba42 wrote:

I was finally able to reproduce this in a Katello 3.18.2.1 development environment. Hopefully this will allow me to make some progress investigating this.

Neat :) I'll leave my test environment up though until you don't need it anymore. Just drop me a line at that point.

I was able to find one difference between my Pulp development environment (where this sync is successful) and my Katello development environment (where it is not): Katello defaults to using Mirror on Sync: Yes while plain Pulp uses Mirror on Sync: No.

Perhaps someone could try using Mirror on Sync: No in Katello to check if that works as a workaround for this problem until a proper fix is available?

Using Mirror on Sync: No does not appear to make a difference.

still the same error with "Mirror on Sync: No" / Katello 4.0

I am 80% sure I have found the bug. It looks like the relative_path for translation files is not being built correctly.

The real question is why this did not fail more often... I will start implementing and testing a fix tomorrow.

many thanks for the efforts

Is there a way to implement the fix on kattelo (3.18) (pulp-deb-plugins-1.10.2) yet?

I've updated the following row at synchronizing.py but that doesn't fix it on my end?

relative_path = os.path.join(os.path.dirname(release_file.relative_path), path)

stfn wrote:

many thanks for the efforts

Is there a way to implement the fix on kattelo (3.18) (pulp-deb-plugins-1.10.2) yet?

I am working on getting a fixed version of pulp_deb into the repository used by Katello (for 3.18 and newer). However there is some worry that this might require newer API bindings on the Katello side, which is why this has not yet happened. Once the following PR is merged, Katello installations should have access to updated RPM packages: https://github.com/theforeman/pulpcore-packaging/pull/144

I've updated the following row at synchronizing.py but that doesn't fix it on my end?

relative_path = os.path.join(os.path.dirname(release_file.relative_path), path)

If you have manually patched this on your system (for me the file is at /usr/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py), that should fix the problem, provided the change has fully taken effect, and there is no interference from objects created by previous failed sync attempts. Things you could try to ensure this is the case:

• Restart pulp services: systemctl restart pulp*
• Restart Foreman/Katello services: foreman-maintain service restart
• If that still does not fix the problem, perhaps running a Pulp orphan cleanup might help (Not sure what the command to do this is, but I believe there is a rake task for it).

I am not sure which of these steps is strictly necessary. I am pretty sure the fix itself works since I tested it on a system that reliably reproduced the error without it.

stfn wrote:

I've applied the fix suggested, but once you try to sync/create a repo after that, the following error occurs.

I guess Katello should also update their code.

Did you update to a newer pulp_deb version, or did you just manually patch the one line in /usr/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py?

Edit: Also check if all your pulp services are running properly, the error suggests they might not have started correctly. (systemctl status pulp*)

quba42 wrote:

Did you update to a newer pulp_deb version, or did you just manually patch the one line in /usr/lib/python3.6/site-packages/pulp_deb/app/tasks/synchronizing.py?

I've manually patched it.

Transaction ID : 31
Begin time     : Tue Apr 20 11:38:05 2021
Package        : pulp-deb-plugins-1.10.2-1.el7.noarch
State          : Install
Size           : 232,774
Build host     : koji.katello.org
Build time     : Thu Oct 29 21:25:39 2020
Packager       : Koji
Vendor         : Koji
License        : GPLv2
URL            : https://github.com/pulp/pulp_deb
Source RPM     : pulp-deb-1.10.2-1.el7.src.rpm
Commit Time    : Wed Mar  4 13:00:00 2020
Committer      : Evgeni Golov - 1.10.1-1
Reason         : user
Command Line   : -d 0 -e 0 -y install pulp-deb-plugins
From repo      : pulp
Installed by   : root <root>

where can I get the patched version?

Edit: Also check if all your pulp services are running properly, the error suggests they might not have started correctly. (systemctl status pulp*)

after reboot everything is running, and on a later stage some of the workeras start failing.. for example

● pulpcore-worker@3.service - Pulp RQ Worker
   Loaded: loaded (/etc/systemd/system/pulpcore-worker@.service; enabled; vendor preset: disabled)
   Active: activating (auto-restart) (Result: exit-code) since Tue 2021-04-20 15:12:23 CEST; 963ms ago
  Process: 12061 ExecStart=/usr/libexec/pulpcore/rq worker -w pulpcore.tasking.worker.PulpWorker -c pulpcore.rqconfig --disable-job-desc-logging (code=exited, status=1/FAILURE)
 Main PID: 12061 (code=exited, status=1/FAILURE)
   CGroup: /system.slice/system-pulpcore\x2dworker.slice/pulpcore-worker@3.service

Apr 20 15:12:23 localhost systemd[1]: Unit pulpcore-worker@3.service entered failed state.
Apr 20 15:12:23 localhost systemd[1]: pulpcore-worker@3.service failed.

I mean I've tried to get the latest source from here and follow the instructions, but these assume that you have installed the pulpcore from source as well, which is not my case, so it doesn't work this way, I guess just waiting for an rpm package would be the easiest option.

https://github.com/pulp/pulp_deb https://docs.pulpproject.org/pulp_deb/installation.html