Task #1200
closed
Story #1150: As a user, I can lazily fetch repositories
Make all plugin sync importers and plugin distributors lazy aware
10%
Description
Each repo has 'lazy passive' or 'lazy active' enabled by setting a key named 'lazy_sync' whose value takes on either 'active' or 'passive'. These configs are read by each plugin's implementation of the 'sync' importer.
Plugin updates¶
The following two things will need to be done in each plugins. <sad face>
- Build "the unit catalog"
- If lazy_sync is set at to any value, sync the metadata and do not download non-metadata units. This will need to be done in each plugin's implementation of the sync handler. FYI, the plugin specific code is called by the platform sync task here.
Platform updates¶
The handling of the lazy_sync = active case can be 100% in platform code. The task it is calling (#1199) is plugin agnostic, and also all it needs to do is dispatch the task and save the spawned task on the TaskResult for the sync. I recommend doing this here. One important point is that the publish spawned task needs to be listed before the generic #1199 download task. If it is not done in this order then the CLI won't show sync, then publish, then the status of the generic #1199. Instead it would should sync, a long delay while #1199 completes, and then the publish which on a large repo sync would not be a great CLI experience.
Related issues
Updated by bmbouter over 8 years ago
Should the 'lazy active' case publish the same as 'lazy passive' and have the asynchronous download task provided by #1199 kicked off during the publish operation?
If so can the publish distributor even access this given that these config values are stored in the sync importer's config?
Updated by bmbouter over 8 years ago
- Blocked by Task #1199: Add a generic task for a lazy-sync "active" that the plugins can dispatch added
Updated by bmbouter over 8 years ago
bmbouter wrote:
Should the 'lazy active' case publish the same as 'lazy passive' and have the asynchronous download task provided by #1199 kicked off during the publish operation?
If so can the publish distributor even access this given that these config values are stored in the sync importer's config?
After talking with bcourt we've determined this can be kicked off as a spawned task after a sync. It's not related to a publish at all and can occur concurrently. The publish will publish its links the same regardless of the amount of units the asynchronous task has done. I've updated the story to reflect this understanding.
Added by jortel@redhat.com over 8 years ago
Added by jortel@redhat.com over 8 years ago
Revision 039329ce | View on GitHub
ref #1200 - add lazy properties to import configuration.
Updated by jortel@redhat.com over 8 years ago
Added importer configuration properties for lazy. https://github.com/pulp/pulp/pull/2085
Updated by jortel@redhat.com over 8 years ago
The celery task needs to skip files that have already been downloaded.
Updated by bmbouter about 5 years ago
- Status changed from NEW to CLOSED - WONTFIX
Updated by bmbouter about 5 years ago
Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.
.
ref #1200 - add lazy properties to import configuration.