https://pulp.plan.io/https://pulp.plan.io/favicon.ico2015-10-21T16:20:57ZPulpPulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=66892015-10-21T16:20:57Zjortel@redhat.comjortel@redhat.com
<ul><li><strong>Subject</strong> changed from <i>Update orphan purge to clean up catalog and downloads collections.</i> to <i>Update orphan purge to clean up catalog.</i></li></ul> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=67672015-10-29T14:50:42Zjortel@redhat.comjortel@redhat.com
<ul><li><strong>Subject</strong> changed from <i>Update orphan purge to clean up catalog.</i> to <i>Update orphan purge to clean up catalog and deferred downloads.</i></li></ul> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=67692015-10-29T14:53:41Zjortel@redhat.comjortel@redhat.com
<ul><li><strong>Subject</strong> changed from <i>Update orphan purge to clean up catalog and deferred downloads.</i> to <i>Purge the lazy catalog and deferred downloads when importer is removed.</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/6769/diff?detail_id=6778">diff</a>)</li></ul> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=70592015-11-16T18:38:21Zjcline@redhat.comjcline@redhat.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-11 priority-6 priority-default closed child" href="/issues/1339">Task #1339</a>: Purge the lazy catalog and deferred downloads when a orphaned unit is removed.</i> added</li></ul> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=70612015-11-16T19:11:49Zjcline@redhat.comjcline@redhat.com
<ul></ul><p>I propose that deferred downloads are never purged. If we choose to never purge deferred downloads, the following will occur:</p>
<ul>
<li>All deferred downloads will eventually be removed during the regularly dispatched ``download_deferred`` task.</li>
</ul>
<ul>
<li>In every case, when the deferred download is processed, it is removed from the database.</li>
</ul>
<ul>
<li>In the event that there is a deferred download entry that does not correspond to any catalog entries, the deferred download is simply skipped.</li>
</ul>
<ul>
<li>In the event that there <strong>are</strong> catalog entries (from another importer), we should download the unit.</li>
</ul>
<ul>
<li>In the event that there is no content unit to go with a deferred download (because an orphan purge occurred before the ``download_deferred`` task could process the entry), that deferred download should be skipped.</li>
</ul>
<p>The one downside to this approach that I see is that we may end up downloading content (from Squid) for orphaned units. This only occurs if a client has recently requested the orphaned unit.</p>
<p>On the other hand, choosing to purge deferred downloads when an importer is removed is, I think, tricky to handle since we would have to know what deferred downloads correspond to units that are managed by the given importer. Certainly doable, but recall that deferred downloads are unique on (unit_id, unit_type_id) only, so if another importer can also get that content unit, there won't be an entry in the DeferredDownload collection until the Squid cache expires and we re-download that unit (using the other importer). We could change the model to include the importer id, but then we would have duplicate entries.</p> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=70712015-11-16T20:39:24Zjortel@redhat.comjortel@redhat.com
<ul></ul><p>I agree, the deferred downloads should not be purged when an importer is removed. This task does not reflect current thinking. However, I do think the catalog should be purged. Since there is a race condition with downloads tasks and orphan purging, I also think it makes sense to not purge the deferred downloads and let the download tasks do it naturally. The download tasks will need to discard deferred downloads when the unit is not-found anyway.</p> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=70722015-11-16T22:12:34Zjcline@redhat.comjcline@redhat.com
<ul><li><strong>Subject</strong> changed from <i>Purge the lazy catalog and deferred downloads when importer is removed.</i> to <i>Purge the lazy catalog when importer is removed.</i></li><li><strong>Status</strong> changed from <i>NEW</i> to <i>POST</i></li><li><strong>Assignee</strong> set to <i>jcline@redhat.com</i></li></ul><p><a href="https://github.com/pulp/pulp/pull/2169" class="external">https://github.com/pulp/pulp/pull/2169</a></p> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=70992015-11-18T21:13:50Zjcline@redhat.comjcline@redhat.com
<ul><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=71572015-11-23T18:40:36ZAnonymous
<ul><li><strong>Status</strong> changed from <i>POST</i> to <i>MODIFIED</i></li></ul><p>Applied in changeset <a class="changeset" title="The lazy catalog is now purged when an importer is deleted. This commit adds a ``pre_delete`` si..." href="https://pulp.plan.io/projects/pulp/repository/pulp/revisions/2956504bc8e765c97a1c8cfeb660c999c0efa576">pulp|2956504bc8e765c97a1c8cfeb660c999c0efa576</a>.</p> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=71582015-11-23T18:41:55Zjcline@redhat.comjcline@redhat.com
<ul><li><strong>Status</strong> changed from <i>MODIFIED</i> to <i>POST</i></li></ul><p>This got merged into the ``lazy-content`` branch, so it's not MODIFIED just yet.</p> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=76132016-01-04T15:20:09Zjcline@redhat.comjcline@redhat.com
<ul><li><strong>Status</strong> changed from <i>POST</i> to <i>MODIFIED</i></li></ul> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=88252016-02-11T20:33:00Zrbarlow
<ul><li><strong>Status</strong> changed from <i>MODIFIED</i> to <i>5</i></li><li><strong>Platform Release</strong> set to <i>2.8.0</i></li></ul> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=101402016-03-23T18:48:01Zdkliban@redhat.com
<ul><li><strong>Status</strong> changed from <i>5</i> to <i>CLOSED - CURRENTRELEASE</i></li></ul> Pulp - Task #1327: Purge the lazy catalog when importer is removed.https://pulp.plan.io/issues/1327?journal_id=394992019-04-15T20:42:08Zbmbouterbmbouter@redhat.com
<ul><li><strong>Tags</strong> <i>Pulp 2</i> added</li></ul>