Project

Profile

Help

Issue #2606

closed

Fix the ability to forge importer to mirror upstream repository on sync

Added by ttereshc about 7 years ago. Updated about 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
2.12.2
OS:
Triaged:
Yes
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Sprint 16
Quarter:

Description

There is a need to be able to mirror Puppet repositories, including the optional ability to remove any local modules that no longer are present in the remote repo. This is called the "remove_missing" setting in the yum importer.

Currently Katello has a number of steps to do in order to achieve that if when the sync is not the first one:
- unassociate all the puppet modules from repo
- perform sync
- clean orphans

When multiple Katello syncs run in parallel, it leads to race condition between one Pulp sync task and the orphan removal task. So that sync notices nothing but during publish one can see the error like this:

pulp: pulp_puppet.plugins.distributors.publish:ERROR: (12867-29760) IOError: [Errno 2] No such file or directory: u'/var/lib/pulp/content/units/puppet_module/4c/052a353708dab8210898dd297f74cf9e4db4e896ae2bbca487719f02e62a3d/smals-mid_modules_wls_12-1.0.54.tar.gz'

Adding Fixing the remove_missing option will solve the issue because the described steps won't be needed anymore. It will have the side benefit of reducing the amount of work that needs to happen during a "smart proxy" sync.

This should be done for the forge importer case.

Actions #1

Updated by ttereshc about 7 years ago

  • Subject changed from Add a possibility to importer to mirror upstream repository on sync to Add an ability to importer to mirror upstream repository on sync
Actions #2

Updated by mhrivnak about 7 years ago

  • Subject changed from Add an ability to importer to mirror upstream repository on sync to Add ability to forge importer to mirror upstream repository on sync
  • Description updated (diff)
  • Groomed changed from No to Yes
  • Sprint Candidate changed from No to Yes
Actions #3

Updated by ttereshc about 7 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ttereshc
  • Sprint/Milestone set to 33
Actions #4

Updated by mhrivnak about 7 years ago

  • Sprint/Milestone changed from 33 to 34
Actions #5

Updated by ttereshc about 7 years ago

  • Tracker changed from Story to Issue
  • Subject changed from Add ability to forge importer to mirror upstream repository on sync to Fix the ability to forge importer to mirror upstream repository on sync
  • Description updated (diff)
  • Severity set to 2. Medium
  • Triaged set to No
Actions #6

Updated by ttereshc about 7 years ago

  • Priority changed from Normal to High
  • Triaged changed from No to Yes
Actions #7

Updated by ttereshc about 7 years ago

  • Status changed from ASSIGNED to POST

Added by ttereshc about 7 years ago

Revision 034ef746 | View on GitHub

Fix remove_missing option and unnecessary downloads for forge importer

Fix remove_missing option for the forge and directory importers. Do not re-download units if they are in DB and on a disk for the forge importer.

closes #2606 https://pulp.plan.io/issues/2606

Actions #9

Updated by ttereshc about 7 years ago

  • Status changed from POST to MODIFIED
Actions #10

Updated by bizhang about 7 years ago

  • Platform Release set to 2.12.2
Actions #11

Updated by bizhang about 7 years ago

  • Status changed from MODIFIED to 5
Actions #12

Updated by bizhang about 7 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #14

Updated by bmbouter about 6 years ago

  • Sprint set to Sprint 16
Actions #15

Updated by bmbouter about 6 years ago

  • Sprint/Milestone deleted (34)
Actions #16

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF