Project

Profile

Help

Pulp3 Plugin Brainstorming » History » Revision 3

Revision 2 (bmbouter, 08/15/2017 08:02 PM) → Revision 3/7 (bmbouter, 08/15/2017 08:02 PM)

----- 

 Content Unit Creator - Downloads and creates objects 

   - Saves all necessary units to the db after downloading all artifacts 
   - Race condition handling for db around object creation 
   - Transactional support around unit creation to ensure always creating a complete content unit or none 

 open questions: should this associate each unit as its made? 

 ----- 

 Content Unit Downloader - Parallelized downloads, but it emits a content unit only when all files are downloaded 

   - manages the asyncio loop for you 
   - abstracts the collecting of downloads for a given content unit 
   - driven by a simple loop 

 ----- 

 Concurrent Downloader - Downloads any number of artifacts in parallel 

   - provides digest and size validation 
   - yields a file that is written to a temp area and all digests which were computed during download 
   - you provide the asyncio loop 
   - constrains the number of connections 
   - share resources such as sessions,connection pools and auth tokens 

 ----- 

 Synchronous Syncronous Downloader - download a single file synchronously synchornously 

   - blocking call so one download at a time 
   - provides digest and size validation 
   - yields a file that is written to a temp area and all digests which were computed during download 

 -----