Pulp3 Plugin Brainstorming » History » Revision 6
Revision 5 (bmbouter, 08/21/2017 05:15 PM) → Revision 6/7 (dkliban@redhat.com, 08/21/2017 06:37 PM)
----- Content Unit Creator - Downloads and creates objects - Process Processes content units in slices to limit the number of in-memory objects - Determine Determines which content units are already present in Pulp - Save new content units not already in Pulp - For immediate download policy, determine determines which artifacts in a slice are already present in Pulp. - Create Deferred Artifacts - Create Content Artifacts - Create new Content units not already in Pulp - For immediate download policy, download Download all artifacts for each content unit. unit - For immediate download policy, save Content Artifacts, Deferred Artifacts, and Artifacts Save all necessary units to the db after downloading all artifacts. - Save Content unit to the db. artifacts - Handles race conditions for db around object creation - Transactional support around unit creation to ensure always creating a complete content unit or none open questions: should this associate each unit as its made? ----- Content Unit Downloader - Parallelized downloads, but it emits a content unit only when all files are downloaded - manages the asyncio loop for you - abstracts the collecting of downloads for a given content unit - driven by a simple loop ----- Concurrent Downloader - Downloads any number of artifacts in parallel - provides digest and size validation - yields a file that is written to a temp area and all digests which were computed during download - you provide the asyncio loop - constrains the number of connections - share resources such as sessions,connection pools and auth tokens ----- Synchronous Downloader - download a single file synchronously - blocking call so one download at a time - provides digest and size validation - yields a file that is written to a temp area and all digests which were computed during download -----
.