Project

Profile

Help

Task #3663

closed

Investigate orphan cleanup performance

Added by ttereshc almost 6 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.16.2
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Sprint 38
Quarter:

Description

If there are many orphans to be removed, the orphan cleanup task can take several hours.

The current orphan cleanup task performs everything without concurrency.
We can introduce a new orphan cleanup task that breaks up the list of orphans into groups and performs cleanup on them in parallel.

A good start is to cprofile this task to see where the bottleneck is and to determine if parallel execution will help.

Some facts to consider:
- orphan cleanup task is a task without resource reservation
- currently multiple orphan cleanup tasks can run at the same time (with the same list of orphans) and that increases mongo cpu usage and significantly slows down performance. See #3045.

The outcome of this task should be another task with detailed proposal on how to improve orphan cleanup performance.
As a compromise, if improving orphan cleanup task is not an option, consider providing an orphan cleanup script, which users can run in case they have a lot of orphans.

Also available in: Atom PDF