Project

Profile

Help

Issue #3887

closed

Restarting pulp_celerybeat weekly or so causes pulp.server.maintenance.monthly to not get scheduled

Added by ttereshc over 5 years ago. Updated about 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
2.17.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Documentation, Pulp 2
Sprint:
Sprint 41
Quarter:

Description

Symptoms

No signs of `pulp.server.maintenance.monthly` in the logs for the last 30 days.
If task is triggered manually, it works properly and orphaned applicability profiles got cleaned up.

Impact

Having a lot of orphaned applicability profiles triggers a lot of no-op applicability tasks, it can slow down the whole system if there are too many of those tasks. E.g. A repo with many bound consumers (30K unique profiles) will trigger 3K tasks.

Root Cause

Pulp users Celery's periodic tasks feature for its dispatching. The monthly maintenance is dispatched every 30 days. Unfortunately Celery measues this as 30 days since the pulp_celerybeat process started. See the celery docs This means that if you restart your pulp_celerybeat every now and then, you'll end up with a lot of orphaned applicability profiles.

Resolution

The recommendation is to use cron to run a script monthly like:

from pulp.server.maintenance.monthly import queue_monthly_maintenance
queue_monthly_maintenance.apply_async()

Related issues

Related to Pulp - Issue #3908: Add pulp-maintenance package CLOSED - CURRENTRELEASEbmbouterActions

Also available in: Atom PDF