Issue #7152
closedmigration uses a lot of memory with a lot of rpms
Description
Attempting a migration with ~51K rpms synced into my pulp2 seems to cause the migration proccess to use more memory than expected. after initially getting some OOMs, i shut down some things and investigated and saw it using quite a bit of memory:
pulp 12860 78.8 46.7 6448656 5778552 ? Rl 20:56 7:20 /usr/bin/python3 /usr/bin/rq worker -w pulpcore.tasking.worker.PulpWorker --pid=/var/run/pulpcore-worker-2/reserved-resource-worker-2.pid -c pulpcore.rqconfig --disable-job-desc-loggin
I'm not sure exactly which part of the migration is happening, but i suspect its when importing rpms, because after the OOM, i saw:
pulpcore=# select count(*) from pulp_2to3_migration_pulp2content where pulp2_content_type_id = 'rpm' and pulp3_content_id is NULL;
count
-------
2000
(1 row)
pulpcore=# select count(*) from pulp_2to3_migration_pulp2content where pulp2_content_type_id = 'rpm' and pulp3_content_id is not NULL;
count
-------
48340
(1 row)
Updated by dalley over 4 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to dalley
Added by dalley over 4 years ago
Added by dalley over 4 years ago
Revision ddc28860 | View on GitHub
Fix high memory consumption issues
Added by dalley over 4 years ago
Revision ddc28860 | View on GitHub
Fix high memory consumption issues
Updated by dalley over 4 years ago
- Status changed from ASSIGNED to POST
Updated by dalley over 4 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulp:pulp-2to3-migration|ddc28860d5ac3045369dd8468f79867ec4329b72.
Added by dalley over 4 years ago
Revision 7d581282 | View on GitHub
Fix high memory consumption issues
closes: #7152 https://pulp.plan.io/issues/7152 (cherry picked from commit ddc28860d5ac3045369dd8468f79867ec4329b72)
Added by dalley over 4 years ago
Revision 7d581282 | View on GitHub
Fix high memory consumption issues
closes: #7152 https://pulp.plan.io/issues/7152 (cherry picked from commit ddc28860d5ac3045369dd8468f79867ec4329b72)
Added by dalley over 4 years ago
Revision 7d581282 | View on GitHub
Fix high memory consumption issues
closes: #7152 https://pulp.plan.io/issues/7152 (cherry picked from commit ddc28860d5ac3045369dd8468f79867ec4329b72)
Updated by ttereshc over 4 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Fix high memory consumption issues
closes: #7152 https://pulp.plan.io/issues/7152