Actions
Issue #8228
closedTask #7884: Move the pulp_installer Vagrant tests off Travis
Pulp Connection Timed Out on slow emulated machines
Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
Installer - Moved to GitHub issues
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Master
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:
Description
When you run pulp_installer 3.10.0 against a very slow machine, such as a Qemu emulated machine with 2 CPU cores (on top of GHA CI, or on my Core i7-8650U system), trying to do any request to the API may result in a "Connection Timed Out" for every single request.
Even the command curl http://127.0.0.1/
would exhibit this.
logs would look like:
Jan 28 20:53:15 pulp3-source-fedora32.lan systemd[1]: Started Pulp WSGI Server.
Jan 28 20:53:19 pulp3-source-fedora32.lan gunicorn[21755]: [2021-01-28 20:53:19 +0000] [21755] [INFO] Starting gunicorn 20.0.4
Jan 28 20:53:19 pulp3-source-fedora32.lan gunicorn[21755]: [2021-01-28 20:53:19 +0000] [21755] [INFO] Listening at: http://127.0.0.1:24817 (21755)
Jan 28 20:53:19 pulp3-source-fedora32.lan gunicorn[21755]: [2021-01-28 20:53:19 +0000] [21755] [INFO] Using worker: sync
Jan 28 20:53:20 pulp3-source-fedora32.lan gunicorn[21774]: [2021-01-28 20:53:20 +0000] [21774] [INFO] Booting worker with pid: 21774
Jan 28 20:53:50 pulp3-source-fedora32.lan gunicorn[21755]: [2021-01-28 20:53:50 +0000] [21755] [CRITICAL] WORKER TIMEOUT (pid:21774)
Jan 28 20:53:50 pulp3-source-fedora32.lan gunicorn[21774]: [2021-01-28 20:53:50 +0000] [21774] [INFO] Worker exiting (pid: 21774)
Jan 28 20:53:51 pulp3-source-fedora32.lan gunicorn[21778]: [2021-01-28 20:53:51 +0000] [21778] [INFO] Booting worker with pid: 21778
Jan 28 20:54:21 pulp3-source-fedora32.lan gunicorn[21755]: [2021-01-28 20:54:21 +0000] [21755] [CRITICAL] WORKER TIMEOUT (pid:21778)
Jan 28 20:54:21 pulp3-source-fedora32.lan gunicorn[21778]: [2021-01-28 20:54:21 +0000] [21778] [INFO] Worker exiting (pid: 21778)
I discovered that by passing to gunicorn (pulpcore-api and pulpcore-content) "--timeout 90" (in the systemd unit files), it fixes this issue. The 1st request may take a while, but later requests would return almost immediately.
Actions
Problem: Pulp Connection Timed Out on slow emulated machines
Solution: Raise the the Pulp server's gunicorn worker timeout to 90 seconds.
fixes: #8228 Pulp Connection Timed Out on slow emulated machines https://pulp.plan.io/issues/8228