Actions
Issue #7152
closedmigration uses a lot of memory with a lot of rpms
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Quarter:
Description
Attempting a migration with ~51K rpms synced into my pulp2 seems to cause the migration proccess to use more memory than expected. after initially getting some OOMs, i shut down some things and investigated and saw it using quite a bit of memory:
pulp 12860 78.8 46.7 6448656 5778552 ? Rl 20:56 7:20 /usr/bin/python3 /usr/bin/rq worker -w pulpcore.tasking.worker.PulpWorker --pid=/var/run/pulpcore-worker-2/reserved-resource-worker-2.pid -c pulpcore.rqconfig --disable-job-desc-loggin
I'm not sure exactly which part of the migration is happening, but i suspect its when importing rpms, because after the OOM, i saw:
pulpcore=# select count(*) from pulp_2to3_migration_pulp2content where pulp2_content_type_id = 'rpm' and pulp3_content_id is NULL;
count
-------
2000
(1 row)
pulpcore=# select count(*) from pulp_2to3_migration_pulp2content where pulp2_content_type_id = 'rpm' and pulp3_content_id is not NULL;
count
-------
48340
(1 row)
Actions
Fix high memory consumption issues
closes: #7152 https://pulp.plan.io/issues/7152