As a user, I can specify the connection_limit on any Remote
The ArtifactDownloader has a max_downloads restrictor. This is an identical feature to the connection limiting offered with aiohttp itself. Specifically aiohttp offers two features with its TCPConnector:
From their docs
- limit (int) – total number simultaneous connections. If limit is None the connector has no limit (default: 100).
Is this really a duplication?¶
Yes, it is verified via wireshark analysis. A max_downloader=20 and a limit=20 for the same number of TCP connections to the server.
Add connection_limit to the Remote like the rest of the attributes, e.g. proxy settings.
connection_limit in Pulp maps to TCPConnector.limit in aiohttp
What are the right default to Pulp¶
Like Pulp2, we should default connection_limit=5. If unset on a remote, a user will receive these default values.
How to Implement¶
This configuration should be read from the Remote and applied to the session by the Factory just like how the other settings are done. Since an unset value should provide default even if unset, the default actually like in the Factory code. That is here
Add connection_limit to a Remote
limitoption to limit a total number of TCP connections, instead of using
closes #4040 https://pulp.plan.io/issues/4040