Project

Profile

Help

Issue #7152

closed

migration uses a lot of memory with a lot of rpms

Added by jsherril@redhat.com over 4 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Quarter:

Description

Attempting a migration with ~51K rpms synced into my pulp2 seems to cause the migration proccess to use more memory than expected. after initially getting some OOMs, i shut down some things and investigated and saw it using quite a bit of memory:

pulp     12860 78.8 46.7 6448656 5778552 ?     Rl   20:56   7:20 /usr/bin/python3 /usr/bin/rq worker -w pulpcore.tasking.worker.PulpWorker --pid=/var/run/pulpcore-worker-2/reserved-resource-worker-2.pid -c pulpcore.rqconfig --disable-job-desc-loggin

I'm not sure exactly which part of the migration is happening, but i suspect its when importing rpms, because after the OOM, i saw:

pulpcore=# select count(*)  from pulp_2to3_migration_pulp2content where  pulp2_content_type_id = 'rpm' and pulp3_content_id is NULL;
 count 
-------
  2000
(1 row)

pulpcore=# select count(*)  from pulp_2to3_migration_pulp2content where  pulp2_content_type_id = 'rpm' and pulp3_content_id is not NULL;
 count 
-------
 48340
(1 row)

Actions #1

Updated by dalley over 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley

Added by dalley over 4 years ago

Revision ddc28860 | View on GitHub

Fix high memory consumption issues

closes: #7152 https://pulp.plan.io/issues/7152

Added by dalley over 4 years ago

Revision ddc28860 | View on GitHub

Fix high memory consumption issues

closes: #7152 https://pulp.plan.io/issues/7152

Added by dalley over 4 years ago

Revision ddc28860 | View on GitHub

Fix high memory consumption issues

closes: #7152 https://pulp.plan.io/issues/7152

Actions #2

Updated by dalley over 4 years ago

  • Status changed from ASSIGNED to POST
Actions #3

Updated by dalley over 4 years ago

  • Status changed from POST to MODIFIED

Added by dalley over 4 years ago

Revision 7d581282 | View on GitHub

Fix high memory consumption issues

closes: #7152 https://pulp.plan.io/issues/7152 (cherry picked from commit ddc28860d5ac3045369dd8468f79867ec4329b72)

Added by dalley over 4 years ago

Revision 7d581282 | View on GitHub

Fix high memory consumption issues

closes: #7152 https://pulp.plan.io/issues/7152 (cherry picked from commit ddc28860d5ac3045369dd8468f79867ec4329b72)

Added by dalley over 4 years ago

Revision 7d581282 | View on GitHub

Fix high memory consumption issues

closes: #7152 https://pulp.plan.io/issues/7152 (cherry picked from commit ddc28860d5ac3045369dd8468f79867ec4329b72)

Actions #4

Updated by ttereshc over 4 years ago

  • Sprint/Milestone set to 0.2.0
Actions #5

Updated by ttereshc over 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF