Project

Profile

Help

Refactor #7352

closed

Provide a better abstraction for customizing downloader behavior

Added by dalley over 3 years ago. Updated over 2 years ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Wishlist
Sprint:
Quarter:

Description

Ticket moved to GitHub: "pulp/pulpcore/1927":https://github.com/pulp/pulpcore/issues/1927


Customization of downloader behavior is currently achieved by subclassing one of the downloader types from the Plugin API and then modifying the get_downloader() method and downloader_factory property on the Remote (sub)class.

A limitation of this abstraction is that while the get_downloader() method accepts **kwargs to pass down to the (possibly customized) downloader, it will pass all kwargs to any downloader without discriminating by the type of downloader. Therefore it's difficult to send a kwarg to be used on a customized HTTP downloader without it being also sent to other types of downloaders such as FileDownloader which are not expecting it.

This could be worked around by additionally modifying the DownloaderFactory, but it's currently suboptimal because:

  • The methods you'd want to override are underscore-prefixed (private)
  • It would create extra coupling and interdependence between the classes; not the end of the world, but kind of messy

We could probably benefit from revisiting the design and incorporate some of our experience with customizing downloader behavior in various plugins.


Related issues

Related to RPM Support - Issue #7342: file:// repo syncing fails with unexpected keyword argument 'silence_errors_for_response_status_codes'CLOSED - CURRENTRELEASEdalleyActions
Related to Pulp - Issue #9003: Repo sync fails with '__init__() got an unexpected keyword argument 'max_retries''CLOSED - CURRENTRELEASEdalleyActions
Related to Pulp - Task #6965: Remote.get_downloader() needs to include remote in kwargs for Factory.build()CLOSED - DUPLICATE

Actions
Related to Pulp - Story #7464: As a plugin writer, I can subclass and re-use ArtifactDownloaderCLOSED - DUPLICATE

Actions
Related to Pulp - Story #9593: As a plugin writer, I want a downloader which streams content and doesn't save it on a file systemCLOSED - DUPLICATE

Actions

Also available in: Atom PDF