Project

Profile

Help

Task #1327

closed

Story #1150: As a user, I can lazily fetch repositories

Purge the lazy catalog when importer is removed.

Added by jortel@redhat.com over 8 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.8.0
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Purge the lazy catalog and deferred downloads when importer is removed.


Related issues

Related to Pulp - Task #1339: Purge the lazy catalog and deferred downloads when a orphaned unit is removed.CLOSED - CURRENTRELEASEjcline@redhat.com

Actions
Actions #1

Updated by jortel@redhat.com over 8 years ago

  • Subject changed from Update orphan purge to clean up catalog and downloads collections. to Update orphan purge to clean up catalog.
Actions #2

Updated by jortel@redhat.com over 8 years ago

  • Subject changed from Update orphan purge to clean up catalog. to Update orphan purge to clean up catalog and deferred downloads.
Actions #3

Updated by jortel@redhat.com over 8 years ago

  • Subject changed from Update orphan purge to clean up catalog and deferred downloads. to Purge the lazy catalog and deferred downloads when importer is removed.
  • Description updated (diff)
Actions #4

Updated by jcline@redhat.com over 8 years ago

  • Related to Task #1339: Purge the lazy catalog and deferred downloads when a orphaned unit is removed. added
Actions #5

Updated by jcline@redhat.com over 8 years ago

I propose that deferred downloads are never purged. If we choose to never purge deferred downloads, the following will occur:

  • All deferred downloads will eventually be removed during the regularly dispatched ``download_deferred`` task.
  • In every case, when the deferred download is processed, it is removed from the database.
  • In the event that there is a deferred download entry that does not correspond to any catalog entries, the deferred download is simply skipped.
  • In the event that there are catalog entries (from another importer), we should download the unit.
  • In the event that there is no content unit to go with a deferred download (because an orphan purge occurred before the ``download_deferred`` task could process the entry), that deferred download should be skipped.

The one downside to this approach that I see is that we may end up downloading content (from Squid) for orphaned units. This only occurs if a client has recently requested the orphaned unit.

On the other hand, choosing to purge deferred downloads when an importer is removed is, I think, tricky to handle since we would have to know what deferred downloads correspond to units that are managed by the given importer. Certainly doable, but recall that deferred downloads are unique on (unit_id, unit_type_id) only, so if another importer can also get that content unit, there won't be an entry in the DeferredDownload collection until the Squid cache expires and we re-download that unit (using the other importer). We could change the model to include the importer id, but then we would have duplicate entries.

Actions #6

Updated by jortel@redhat.com over 8 years ago

I agree, the deferred downloads should not be purged when an importer is removed. This task does not reflect current thinking. However, I do think the catalog should be purged. Since there is a race condition with downloads tasks and orphan purging, I also think it makes sense to not purge the deferred downloads and let the download tasks do it naturally. The download tasks will need to discard deferred downloads when the unit is not-found anyway.

Actions #7

Updated by jcline@redhat.com over 8 years ago

  • Subject changed from Purge the lazy catalog and deferred downloads when importer is removed. to Purge the lazy catalog when importer is removed.
  • Status changed from NEW to POST
  • Assignee set to jcline@redhat.com
Actions #8

Updated by jcline@redhat.com over 8 years ago

  • % Done changed from 0 to 100

Added by Jeremy Cline over 8 years ago

Revision 2956504b | View on GitHub

The lazy catalog is now purged when an importer is deleted.

This commit adds a pre_delete signal to the Importer model which handles the removal of catalog entries.

closes #1327

Added by Jeremy Cline over 8 years ago

Revision 2956504b | View on GitHub

The lazy catalog is now purged when an importer is deleted.

This commit adds a pre_delete signal to the Importer model which handles the removal of catalog entries.

closes #1327

Actions #9

Updated by Anonymous over 8 years ago

  • Status changed from POST to MODIFIED
Actions #10

Updated by jcline@redhat.com over 8 years ago

  • Status changed from MODIFIED to POST

This got merged into the ``lazy-content`` branch, so it's not MODIFIED just yet.

Actions #11

Updated by jcline@redhat.com about 8 years ago

  • Status changed from POST to MODIFIED
Actions #12

Updated by rbarlow about 8 years ago

  • Status changed from MODIFIED to 5
  • Platform Release set to 2.8.0
Actions #13

Updated by dkliban@redhat.com about 8 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #14

Updated by bmbouter almost 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF