Project

Profile

Help

Story #4040

closed

As a user, I can specify the connection_limit on any Remote

Added by bmbouter over 5 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 43
Quarter:

Description

Problem

The ArtifactDownloader has a max_downloads restrictor. This is an identical feature to the connection limiting offered with aiohttp itself. Specifically aiohttp offers two features with its TCPConnector:

From their docs

  • limit (int) – total number simultaneous connections. If limit is None the connector has no limit (default: 100).

Is this really a duplication?

Yes, it is verified via wireshark analysis. A max_downloader=20 and a limit=20 for the same number of TCP connections to the server.

Solution

Add connection_limit to the Remote like the rest of the attributes, e.g. proxy settings.

connection_limit in Pulp maps to TCPConnector.limit in aiohttp

What are the right default to Pulp

Like Pulp2, we should default connection_limit=5. If unset on a remote, a user will receive these default values.

How to Implement

This configuration should be read from the Remote and applied to the session by the Factory just like how the other settings are done. Since an unset value should provide default even if unset, the default actually like in the Factory code. That is here


Related issues

Related to Pulp - Story #4039: As a plugin writer, I can specify max_concurrent_downloader and max_content_unit via DeclarativeVersionCLOSED - WONTFIX

Actions

Also available in: Atom PDF