Pulp

The GroupDownloader should be deleted.

What it did¶

The idea with GroupDownloader is that you can create a group of files that belong to content unit and have all the downloads of all the content units happen in parallel.

Why delete it?¶

asyncio.gather(...) provides the same facility in the standard library: https://docs.python.org/3/library/asyncio-task.html#asyncio.gather

Take your N artifacts for your single content type and wrap them with gather() creating a single co-routine to track them. Then for M content units do the same thing, resulting in M co-routines returned by gather(...). You can schedule these M coroutines which will schedule all downloads conncurrently. Also you can wake up with each of the M coroutines finishing, each representing a completed content unit. This is exactly the same feature set as GroupDownloader.

See it in action¶

The ArtifactDownloader stage implements the exact alternate pattern described ^ here: https://github.com/pulp/pulp/blob/master/plugin/pulpcore/plugin/stages/artifact_stages.py#L194

Also remove attach_url_to_exception¶

The attach_url_to_exception decorator only serves the GroupDownloader. It's akward anyway and should also be removed. https://github.com/pulp/pulp/blob/master/plugin/pulpcore/plugin/download/base.py#L29-L55 gather handles exception better than our custom solution did anyway.