Pulp

I propose that deferred downloads are never purged. If we choose to never purge deferred downloads, the following will occur:

• All deferred downloads will eventually be removed during the regularly dispatched ``download_deferred`` task.

• In every case, when the deferred download is processed, it is removed from the database.

• In the event that there is a deferred download entry that does not correspond to any catalog entries, the deferred download is simply skipped.

• In the event that there are catalog entries (from another importer), we should download the unit.

• In the event that there is no content unit to go with a deferred download (because an orphan purge occurred before the ``download_deferred`` task could process the entry), that deferred download should be skipped.

The one downside to this approach that I see is that we may end up downloading content (from Squid) for orphaned units. This only occurs if a client has recently requested the orphaned unit.

On the other hand, choosing to purge deferred downloads when an importer is removed is, I think, tricky to handle since we would have to know what deferred downloads correspond to units that are managed by the given importer. Certainly doable, but recall that deferred downloads are unique on (unit_id, unit_type_id) only, so if another importer can also get that content unit, there won't be an entry in the DeferredDownload collection until the Squid cache expires and we re-download that unit (using the other importer). We could change the model to include the importer id, but then we would have duplicate entries.

I agree, the deferred downloads should not be purged when an importer is removed. This task does not reflect current thinking. However, I do think the catalog should be purged. Since there is a race condition with downloads tasks and orphan purging, I also think it makes sense to not purge the deferred downloads and let the download tasks do it naturally. The download tasks will need to discard deferred downloads when the unit is not-found anyway.