Project

Profile

Help

Refactor #7352

closed

Provide a better abstraction for customizing downloader behavior

Added by dalley over 4 years ago. Updated almost 3 years ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Wishlist
Sprint:
Quarter:

Description

Ticket moved to GitHub: "pulp/pulpcore/1927":https://github.com/pulp/pulpcore/issues/1927


Customization of downloader behavior is currently achieved by subclassing one of the downloader types from the Plugin API and then modifying the get_downloader() method and downloader_factory property on the Remote (sub)class.

A limitation of this abstraction is that while the get_downloader() method accepts **kwargs to pass down to the (possibly customized) downloader, it will pass all kwargs to any downloader without discriminating by the type of downloader. Therefore it's difficult to send a kwarg to be used on a customized HTTP downloader without it being also sent to other types of downloaders such as FileDownloader which are not expecting it.

This could be worked around by additionally modifying the DownloaderFactory, but it's currently suboptimal because:

  • The methods you'd want to override are underscore-prefixed (private)
  • It would create extra coupling and interdependence between the classes; not the end of the world, but kind of messy

We could probably benefit from revisiting the design and incorporate some of our experience with customizing downloader behavior in various plugins.


Related issues

Related to RPM Support - Issue #7342: file:// repo syncing fails with unexpected keyword argument 'silence_errors_for_response_status_codes'CLOSED - CURRENTRELEASEdalleyActions
Related to Pulp - Issue #9003: Repo sync fails with '__init__() got an unexpected keyword argument 'max_retries''CLOSED - CURRENTRELEASEdalleyActions
Related to Pulp - Task #6965: Remote.get_downloader() needs to include remote in kwargs for Factory.build()CLOSED - DUPLICATE

Actions
Related to Pulp - Story #7464: As a plugin writer, I can subclass and re-use ArtifactDownloaderCLOSED - DUPLICATE

Actions
Related to Pulp - Story #9593: As a plugin writer, I want a downloader which streams content and doesn't save it on a file systemCLOSED - DUPLICATE

Actions
Actions #1

Updated by dalley over 4 years ago

  • Related to Issue #7342: file:// repo syncing fails with unexpected keyword argument 'silence_errors_for_response_status_codes' added
Actions #2

Updated by dkliban@redhat.com almost 4 years ago

  • Status changed from NEW to CLOSED - WONTFIX

We will not get to this in the near future.

Actions #3

Updated by dalley almost 4 years ago

  • Status changed from CLOSED - WONTFIX to NEW
  • Tags Pulp-NEXT added
Actions #4

Updated by dalley almost 4 years ago

  • Tags Wishlist added
  • Tags deleted (Pulp-NEXT)
Actions #5

Updated by dalley over 3 years ago

  • Related to Issue #9003: Repo sync fails with '__init__() got an unexpected keyword argument 'max_retries'' added
Actions #6

Updated by dalley over 3 years ago

  • Related to Task #6965: Remote.get_downloader() needs to include remote in kwargs for Factory.build() added
Actions #7

Updated by dalley over 3 years ago

  • Related to Story #7464: As a plugin writer, I can subclass and re-use ArtifactDownloader added
Actions #8

Updated by ttereshc almost 3 years ago

  • Related to Story #9593: As a plugin writer, I want a downloader which streams content and doesn't save it on a file system added
Actions #9

Updated by pulpbot almost 3 years ago

  • Description updated (diff)
  • Status changed from NEW to CLOSED - DUPLICATE

Also available in: Atom PDF