Issue #105
closed
repo unit association does not always use generators, leading to OOMs
Description
Repo association requires the user to pass in a list of filters when working with larger sets of units[1].
There are multiple areas in the repo association code path that use lists instead of generators. For example, RepoUnitAssociationQueryManager.get_units() has a "as_generator" flag that is not used during associations. This is one area that needs to be fixed, but there are a few other areas as well that can cause an OOM if given a large number of units without filtered fields. See [2] for more detail.
Repro info to cause an OOM: sync a repo with >10K units on a machine with 4GB mem, and attempt to associate all rpm units in that repo with a new empty repo without specifying field filters.
[1] http://pulp.readthedocs.org/en/latest/dev-guide/integration/rest-api/content/associate.html
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1158545
Related issues
Updated by cduryee almost 10 years ago
- Status changed from NEW to 7
closing and re-opening as bz for now
Updated by cduryee over 9 years ago
This got re-opened as https://pulp.plan.io/issues/596
Updated by bmbouter over 9 years ago
- Is duplicate of Issue #596: Multiple unit_types in association call causes to fetch everything from source repo added
Updated by bmbouter over 9 years ago
- Status changed from 7 to CLOSED - DUPLICATE
.