Task #3663
Updated by ttereshc almost 6 years ago
If there are many orphans to be removed, the orphan cleanup task can take several hours. The current orphan cleanup task performs everything without concurrency. We can introduce a new orphan cleanup task that breaks up the list of orphans into groups and performs cleanup on them in parallel. A good start is to "cprofile":https://docs.pulpproject.org/dev-guide/debugging.html?#custom-runtime-performance-analysis "cprofile": this task to see where the bottleneck is and to determine if parallel execution will help. Some facts to consider: - orphan cleanup task is a task without resource reservation - currently multiple orphan cleanup tasks can run at the same time (with the same list of orphans) and that increases mongo cpu usage and significantly slows down performance. See #3045. The outcome of this task should be another task with detailed proposal on how to improve orphan cleanup performance. As a compromise, if improving orphan cleanup task is not an option, consider providing an orphan cleanup script, which users can run in case they have a lot of orphans.
.