Story #4040

Updated by bmbouter over 4 years ago

h2. Problem 

 The ArtifactDownloader has a max_downloads restrictor. This is an identical feature to the connection limiting offered with aiohttp itself. Specifically aiohttp offers two features with its TCPConnector: 

 From "their docs": 

 * limit (int) – total number simultaneous connections. If limit is None the connector has no limit (default: 100). 
 * limit_per_host (int) – limit simultaneous connections to the same endpoint. Endpoints are the same if they are have equal (host, port, is_ssl) triple. If limit is 0 the connector has no limit (default: 0). 

 h2. Is this really a duplication? 

 Yes, it is verified via wireshark analysis. A max_downloader=20 and a limit=20 for the same number of TCP connections to the server. 

 h2. Solution 

 Add connection_limit and connection_limit_per_host to the Remote like the rest of the attributes, e.g. proxy settings. 

 connection_limit in Pulp maps to TCPConnector.limit in aiohttp 
 connection_limit_per_host in Pulp maps to TCPConnector.limit_per_host in aiohttp 

 h2. What are the right default to Pulp 

 Like Pulp2, we should default connection_limit=5. We should default connection_limit_per_host to 0 so it's disabled by default. 

 If unset on a remote, a user will receive these default values. 

 h2. How to Implement 

 This configuration should be read from the Remote and applied to the session by the Factory just like how the other settings are done. Since an unset value should provide default even if unset, the default actually like in the Factory code. That is "here":