As a user, I want pulp_celerybeat to survive a brief database outage without having to use the unsafe_autoretry feature.
#3 Updated by bmbouter almost 4 years ago
Here is a recap of some testing I just did against master with unsafe_autoretry=False.
All processes survive a mongodb downtime of 30 seconds already. This is the pymongo default and pymongo doesn't raise the exception until 30 seconds of time has passed. So normal restarts and brief (<30 second) database outages are already handled correctly. After 30 seconds, pulp_celerybeat is the only process that exits and does not recover when mongodb recovers.
Note that if mongodb is down when pulp processes are starting, pulp_celerybeat, pulp_workers, and pulp_resource_manager all will fail to start and will not recover when mongodb recovers. This is an initial connection availability problem and affects almost all of Pulp (except httpd).
Note we identify in the docs that if users want to catch mongodb errors including connection errors, they should turn on unsafe_autoretry.
Please register to edit this issue