Project

Profile

Help

Actions
Send by e-mail Copy link

Refactor #9309

closed

Add support for new memory-efficient createrepo_c parsing method

Added by dalley over 2 years ago. Updated over 2 years ago.

Status:
CLOSED - DUPLICATE
Priority:
High
Assignee:
dalley
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 111
Quarter:

Description

Ticket moved to GitHub: "pulp/pulp_rpm/2296":https://github.com/pulp/pulp_rpm/issues/2296

Files

Download all files
Screenshot from 2021-08-30 00-37-04.png (292 KB) Screenshot from 2021-08-30 00-37-04.png dalley, 08/30/2021 06:40 AM
Screenshot from 2021-08-30 00-40-14.png (363 KB) Screenshot from 2021-08-30 00-40-14.png dalley, 08/30/2021 06:40 AM

Related issues

Has duplicate RPM Support - Issue #9567: More fault tollerant metadata parsingCLOSED - DUPLICATE11/12/2021Actions
Copied to RPM Support - Backport #9313: Backport #9309 "Add support for new memory-efficient createrepo_c parsing method" to 3.14.zCLOSED - DUPLICATEdalley

Actions
Actions
Copy link
 #1

Updated by dalley over 2 years ago

  • Sprint/Milestone set to 3.16.0
Actions
Copy link
 #2

Updated by dalley over 2 years ago

Here's a measurement of the raw overhead of createrepo_c during the parsing of rhel6 repos (so, this doesn't include Pulp at all)

Current parsing strategy (peak: 392mb)

New API (peak: 19mb)

The time taken to parse the metadata also dropped from around 1 min 15 seconds with the current strategy to 13 seconds with the new API.

The old pure-createrepo_c parsing strategy which is disabled in Pulp due to memory consumption peaks at 1.9gb

Actions
Copy link
 #4

Updated by dalley over 2 years ago

  • Copied to Backport #9313: Backport #9309 "Add support for new memory-efficient createrepo_c parsing method" to 3.14.z added
Actions
Copy link
 #6

Updated by dalley over 2 years ago

  • Priority changed from Normal to High
Actions
Copy link
 #7

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 104 to Sprint 105
Actions
Copy link
 #8

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 105 to Sprint 106
Actions
Copy link
 #9

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 106 to Sprint 107
Actions
Copy link
 #10

Updated by dalley over 2 years ago

  • Sprint/Milestone deleted (3.16.0)
Actions
Copy link
 #11

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 107 to Sprint 108
Actions
Copy link
 #12

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 108 to Sprint 109
Actions
Copy link
 #13

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 109 to Sprint 110
Actions
Copy link
 #14

Updated by dalley over 2 years ago

  • Has duplicate Issue #9567: More fault tollerant metadata parsing added
Actions
Copy link
 #15

Updated by rchan over 2 years ago

  • Sprint changed from Sprint 110 to Sprint 111
Actions
Copy link
 #16

Updated by fao89 over 2 years ago

  • Description updated (diff)
  • Status changed from POST to CLOSED - DUPLICATE
Actions
Send by e-mail Copy link

Also available in: Atom PDF