Pulp3 Plugin Brainstorming » History » Sprint/Milestone 3
bmbouter, 08/15/2017 08:02 PM
| 1 | 1 | bmbouter | ----- |
|---|---|---|---|
| 2 | |||
| 3 | Content Unit Creator - Downloads and creates objects |
||
| 4 | |||
| 5 | - Saves all necessary units to the db after downloading all artifacts |
||
| 6 | - Race condition handling for db around object creation |
||
| 7 | - Transactional support around unit creation to ensure always creating a complete content unit or none |
||
| 8 | |||
| 9 | open questions: should this associate each unit as its made? |
||
| 10 | |||
| 11 | ----- |
||
| 12 | |||
| 13 | Content Unit Downloader - Parallelized downloads, but it emits a content unit only when all files are downloaded |
||
| 14 | |||
| 15 | - manages the asyncio loop for you |
||
| 16 | - abstracts the collecting of downloads for a given content unit |
||
| 17 | - driven by a simple loop |
||
| 18 | |||
| 19 | ----- |
||
| 20 | |||
| 21 | Concurrent Downloader - Downloads any number of artifacts in parallel |
||
| 22 | |||
| 23 | - provides digest and size validation |
||
| 24 | - yields a file that is written to a temp area and all digests which were computed during download |
||
| 25 | - you provide the asyncio loop |
||
| 26 | - constrains the number of connections |
||
| 27 | - share resources such as sessions,connection pools and auth tokens |
||
| 28 | |||
| 29 | ----- |
||
| 30 | |||
| 31 | 3 | bmbouter | Synchronous Downloader - download a single file synchronously |
| 32 | 1 | bmbouter | |
| 33 | - blocking call so one download at a time |
||
| 34 | - provides digest and size validation |
||
| 35 | - yields a file that is written to a temp area and all digests which were computed during download |
||
| 36 | |||
| 37 | ----- |
.