Task #1200
closed
Story #1150: As a user, I can lazily fetch repositories
Make all plugin sync importers and plugin distributors lazy aware
Description
Each repo has 'lazy passive' or 'lazy active' enabled by setting a key named 'lazy_sync' whose value takes on either 'active' or 'passive'. These configs are read by each plugin's implementation of the 'sync' importer.
Plugin updates¶
The following two things will need to be done in each plugins. <sad face>
- Build "the unit catalog"
- If lazy_sync is set at to any value, sync the metadata and do not download non-metadata units. This will need to be done in each plugin's implementation of the sync handler. FYI, the plugin specific code is called by the platform sync task here.
Platform updates¶
The handling of the lazy_sync = active case can be 100% in platform code. The task it is calling (#1199) is plugin agnostic, and also all it needs to do is dispatch the task and save the spawned task on the TaskResult for the sync. I recommend doing this here. One important point is that the publish spawned task needs to be listed before the generic #1199 download task. If it is not done in this order then the CLI won't show sync, then publish, then the status of the generic #1199. Instead it would should sync, a long delay while #1199 completes, and then the publish which on a large repo sync would not be a great CLI experience.
Should the 'lazy active' case publish the same as 'lazy passive' and have the asynchronous download task provided by #1199 kicked off during the publish operation?
If so can the publish distributor even access this given that these config values are stored in the sync importer's config?
- Parent issue set to #1150
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
- Blocked by Task #1199: Add a generic task for a lazy-sync "active" that the plugins can dispatch added
bmbouter wrote:
Should the 'lazy active' case publish the same as 'lazy passive' and have the asynchronous download task provided by #1199 kicked off during the publish operation?
If so can the publish distributor even access this given that these config values are stored in the sync importer's config?
After talking with bcourt we've determined this can be kicked off as a spawned task after a sync. It's not related to a publish at all and can occur concurrently. The publish will publish its links the same regardless of the amount of units the asynchronous task has done. I've updated the story to reflect this understanding.
- % Done changed from 0 to 10
The celery task needs to skip files that have already been downloaded.
- Status changed from NEW to CLOSED - WONTFIX
Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.
Also available in: Atom
PDF
.
ref #1200 - add lazy properties to import configuration.