Issue #2606
closedFix the ability to forge importer to mirror upstream repository on sync
Description
There is a need to be able to mirror Puppet repositories, including the optional ability to remove any local modules that no longer are present in the remote repo. This is called the "remove_missing" setting in the yum importer.
Currently Katello has a number of steps to do in order to achieve that if when the sync is not the first one:
- unassociate all the puppet modules from repo
- perform sync
- clean orphans
When multiple Katello syncs run in parallel, it leads to race condition between one Pulp sync task and the orphan removal task. So that sync notices nothing but during publish one can see the error like this:
pulp: pulp_puppet.plugins.distributors.publish:ERROR: (12867-29760) IOError: [Errno 2] No such file or directory: u'/var/lib/pulp/content/units/puppet_module/4c/052a353708dab8210898dd297f74cf9e4db4e896ae2bbca487719f02e62a3d/smals-mid_modules_wls_12-1.0.54.tar.gz'
Adding Fixing the remove_missing option will solve the issue because the described steps won't be needed anymore. It will have the side benefit of reducing the amount of work that needs to happen during a "smart proxy" sync.
This should be done for the forge importer case.
Updated by ttereshc about 7 years ago
- Subject changed from Add a possibility to importer to mirror upstream repository on sync to Add an ability to importer to mirror upstream repository on sync
Updated by mhrivnak about 7 years ago
- Subject changed from Add an ability to importer to mirror upstream repository on sync to Add ability to forge importer to mirror upstream repository on sync
- Description updated (diff)
- Groomed changed from No to Yes
- Sprint Candidate changed from No to Yes
Updated by ttereshc about 7 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to ttereshc
- Sprint/Milestone set to 33
Updated by ttereshc about 7 years ago
- Tracker changed from Story to Issue
- Subject changed from Add ability to forge importer to mirror upstream repository on sync to Fix the ability to forge importer to mirror upstream repository on sync
- Description updated (diff)
- Severity set to 2. Medium
- Triaged set to No
Updated by ttereshc about 7 years ago
- Priority changed from Normal to High
- Triaged changed from No to Yes
Updated by ttereshc about 7 years ago
- Status changed from ASSIGNED to POST
Added by ttereshc about 7 years ago
Updated by ttereshc about 7 years ago
- Status changed from POST to MODIFIED
Applied in changeset 034ef746a8ddb7547766a24da0bcd6563f19a221.
Updated by bizhang about 7 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
Fix remove_missing option and unnecessary downloads for forge importer
Fix remove_missing option for the forge and directory importers. Do not re-download units if they are in DB and on a disk for the forge importer.
closes #2606 https://pulp.plan.io/issues/2606