Pulp3 Plugin Brainstorming » History » Revision 5
Revision 4 (dkliban@redhat.com, 08/21/2017 04:57 PM) → Revision 5/7 (bmbouter, 08/21/2017 05:15 PM)
----- Content Unit Creator - Downloads and creates objects - Processes content units in slices to limit the number of in-memory objects - Determines which content units in a slice are already present in Pulp - Save new content units not already in Pulp - For immediate download policy, determines which artifacts in a slice are already present in Pulp - Download Downloads all artifacts for each content unit - Save Saves all necessary units to the db after downloading all artifacts - Handles race conditions for db around object creation - Transactional support around unit creation to ensure always creating a complete content unit or none open questions: should this associate each unit as its made? ----- Content Unit Downloader - Parallelized downloads, but it emits a content unit only when all files are downloaded - manages the asyncio loop for you - abstracts the collecting of downloads for a given content unit - driven by a simple loop ----- Concurrent Downloader - Downloads any number of artifacts in parallel - provides digest and size validation - yields a file that is written to a temp area and all digests which were computed during download - you provide the asyncio loop - constrains the number of connections - share resources such as sessions,connection pools and auth tokens ----- Synchronous Downloader - download a single file synchronously - blocking call so one download at a time - provides digest and size validation - yields a file that is written to a temp area and all digests which were computed during download -----
.