Project

Profile

Help

Actions
Send by e-mail Copy link

Issue #7483

closed

RPM Copy API encounters timeouts when copying large quanitites of units

Added by iballou over 3 years ago. Updated over 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
dalley
Sprint/Milestone:
3.7.0
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Katello
Sprint:
Sprint 82
Quarter:

Description

With Katello, we are hitting issues with time-outs and request size limits when trying to copy 10s of thousands of content units via the RPM Copy API. To avoid workarounds like implementing our own "batching", it would be best if the RPM Copy API could accept multiple copy requests and do its own batching.

Actions
Copy link
 #1

Updated by dalley over 3 years ago

  • Sprint set to Sprint 81

The likely problem is the following lines: https://github.com/pulp/pulp_rpm/blob/master/pulp_rpm/app/viewsets.py#L317-L318

Wherein each content href is used to individually look up the corresponding content. 10,000 content = at least individual 10,000 queries being made within the span of the single request.

Actions
Copy link
 #2

Updated by dalley over 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley
Actions
Copy link
 #3

Updated by rchan over 3 years ago

  • Sprint changed from Sprint 81 to Sprint 82
Actions
Copy link
 #4

Updated by pulpbot over 3 years ago

  • Status changed from ASSIGNED to POST

Added by dalley over 3 years ago

Revision 61bc0817 | View on GitHub

Fix requests timing out with large copy operations

[nocoverage]

Required PR: https://github.com/pulp/pulpcore/pull/927

closes: #7483 https://pulp.plan.io/issues/7483

Actions
Copy link
 #6

Updated by dalley over 3 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions
Copy link
 #7

Updated by ttereshc over 3 years ago

  • Sprint/Milestone set to 3.7.0
Actions
Copy link
 #8

Updated by dalley over 3 years ago

  • Tracker changed from Story to Issue
  • Subject changed from RPM Copy API should have some sort of batching for large requests to RPM Copy API encounters timeouts when copying large quanitites of units
  • % Done changed from 100 to 0
  • Severity set to 2. Medium
  • Triaged set to No
Actions
Copy link
 #9

Updated by pulpbot over 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions
Send by e-mail Copy link

Also available in: Atom PDF