Project

Profile

Help

Issue #105

repo unit association does not always use generators, leading to OOMs

Added by cduryee almost 7 years ago. Updated over 2 years ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Repo association requires the user to pass in a list of filters when working with larger sets of units[1].

There are multiple areas in the repo association code path that use lists instead of generators. For example, RepoUnitAssociationQueryManager.get_units() has a "as_generator" flag that is not used during associations. This is one area that needs to be fixed, but there are a few other areas as well that can cause an OOM if given a large number of units without filtered fields. See [2] for more detail.

Repro info to cause an OOM: sync a repo with >10K units on a machine with 4GB mem, and attempt to associate all rpm units in that repo with a new empty repo without specifying field filters.

[1] http://pulp.readthedocs.org/en/latest/dev-guide/integration/rest-api/content/associate.html
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1158545


Related issues

Is duplicate of Pulp - Issue #596: Multiple unit_types in association call causes to fetch everything from source repoCLOSED - WONTFIX<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

History

#1 Updated by cduryee almost 7 years ago

  • Status changed from NEW to 7

closing and re-opening as bz for now

#2 Updated by cduryee over 6 years ago

This got re-opened as https://pulp.plan.io/issues/596

#3 Updated by bmbouter over 6 years ago

  • Triaged changed from No to Yes

#4 Updated by bmbouter over 6 years ago

  • Is duplicate of Issue #596: Multiple unit_types in association call causes to fetch everything from source repo added

#5 Updated by bmbouter over 6 years ago

  • Status changed from 7 to CLOSED - DUPLICATE

#6 Updated by bmbouter over 2 years ago

  • Severity set to 2. Medium

#7 Updated by bmbouter over 2 years ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF