Occasional HTTP 403 errors when downloading rpms from on_demand / lazy sync repositories
The streamer is resulting in occasional 403s.
The patched version of the HTTPAdaptor included in the streamer package basically replicates the upstream patch that has been accepted and merged to python-requests proposed/3.0.0. However, the patch to python-requests only addresses part of the problem. It propagates settings to the urllib3 PoolManager. However, the criteria (pool-key) used by the PoolManager to select a connection pool only includes (scheme, host, port). This means connections will be reused even when different certificates are specified. As of this PR, urllib3 was patched to have custom pool keys which (for ssl) includes the certificate path. Good effort. As a result, even when using the latest patched urllib3 (in F25) connections with different certificate paths will not reuse connections when different certificates are specified.
The latest patches to python-requests and urllib3 cannot be used as is with pulp2 because of how Pulp/Nectar handles certificates. Pulp stores certificates in the DB and passes to nectar as PEM strings. Nectar writes them to a temporary file and passes the path to requests. The certificate path will be different every time. This will cause the urllib3 PoolManager to create a new pool for every request which results in never sharing connections.
EL7 has urllib3 1.10
F25 has urllib3 1.15
Cache sessions in the streamer. closes #2677