Network maintenance. Planio will be observing two scheduled maintenance windows this Tuesday, March 2 and Wednesday, March 3 from 02:00 UTC until 06:00 UTC each in order to perform maintenance on access routers in our primary datacenter. Your account might observe short downtimes during these periods up to several minutes at a time.
Issue #1682
Puppet sync starts a new TCP connection for each package rather than pooling
Description
If you start to sync Puppet Forge, you will notice that the log shows that it is creating a connection for each package that it downloads.
$ pulp-admin puppet repo create --repo-id=forge --feed=http://forge.puppetlabs.com
$ time pulp-admin puppet repo sync run --repo-id=forge
You will see many messages like this in the log:
Feb 16 18:04:54 dev pulp[4674]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): forge.puppetlabs.com
Feb 16 18:04:55 dev pulp[4674]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): forge.puppetlabs.com
Feb 16 18:04:56 dev pulp[4674]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): forge.puppetlabs.com
Feb 16 18:04:57 dev pulp[4674]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): forge.puppetlabs.com
Feb 16 18:04:58 dev pulp[4674]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): forge.puppetlabs.com
Feb 16 18:04:59 dev pulp[4674]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): forge.puppetlabs.com
…
I would expect to see it making new connections and then pooling them. 5 connections would look like this in the log:
Feb 16 18:04:54 dev pulp[4674]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): forge.puppetlabs.com
Feb 16 18:04:55 dev pulp[4674]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (2): forge.puppetlabs.com
Feb 16 18:04:56 dev pulp[4674]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (3): forge.puppetlabs.com
Feb 16 18:04:57 dev pulp[4674]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (4): forge.puppetlabs.com
Feb 16 18:04:58 dev pulp[4674]: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (5): forge.puppetlabs.com
If you watch the log during an RPM sync, you will see something similar to the above.
The user-facing issue here is that the sync takes a very long time. It is also not particularly friendly to Puppet forge.
Related issues
History
#1
Updated by rbarlow about 5 years ago
- Has duplicate Story #230: [RFE] The Puppet Forge downloader should download in parallel added
#2
Updated by mhrivnak about 5 years ago
- Subject changed from Puppet sync starts a new HTTP connection for each package rather than pooling to Puppet sync starts a new TCP connection for each package rather than pooling
- Priority changed from Normal to High
- Severity changed from 2. Medium to 1. Low
- Triaged changed from No to Yes
#3
Updated by rbarlow about 5 years ago
- Description updated (diff)
#4
Updated by bmbouter almost 2 years ago
- Status changed from NEW to CLOSED - WONTFIX
Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.
#5
Updated by bmbouter almost 2 years ago
- Tags Pulp 2 added
Please register to edit this issue
.