RPM Copy API encounters timeouts when copying large quanitites of units
With Katello, we are hitting issues with time-outs and request size limits when trying to copy 10s of thousands of content units via the RPM Copy API. To avoid workarounds like implementing our own "batching", it would be best if the RPM Copy API could accept multiple copy requests and do its own batching.
#1 Updated by dalley over 1 year ago
- Sprint set to Sprint 81
The likely problem is the following lines: https://github.com/pulp/pulp_rpm/blob/master/pulp_rpm/app/viewsets.py#L317-L318
Wherein each content href is used to individually look up the corresponding content. 10,000 content = at least individual 10,000 queries being made within the span of the single request.
#8 Updated by dalley over 1 year ago
- Tracker changed from Story to Issue
- Subject changed from RPM Copy API should have some sort of batching for large requests to RPM Copy API encounters timeouts when copying large quanitites of units
- % Done changed from 100 to 0
- Severity set to 2. Medium
- Triaged set to No