Issue #4214
closed
Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE
Status:
CLOSED - CURRENTRELEASE
Description
Submit a large request (> 2.5 MB) to a pulp 2 endpoint and it'll raise the error "body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE". It looks like in this case Katello was posting a rather long list of content units to /pulp/api/v2/repositories/<repo id>/actions/associate/.
Looks like this is a django setting that is by default 2.5 MB.
https://docs.djangoproject.com/en/2.1/ref/settings/#data-upload-max-memory-size
- Is duplicate of Story #4196: As a user, I can upload files in chunks. added
- Status changed from NEW to CLOSED - DUPLICATE
Increasing the max size is a workaround, the more important feature is to chunk large uploads. Not related to uploads
- Description updated (diff)
- Status changed from CLOSED - DUPLICATE to NEW
It wasn't an upload. I've updated the description to add an example of a request.
I think we should either disable DATA_UPLOAD_MAX_MEMORY_SIZE or bump up the size. 2.5 MB seems small especially since Katello will often enumerate a list of ids when performing actions like repo copy.
Maybe we should consider a new config option?
- Is duplicate of deleted (Story #4196: As a user, I can upload files in chunks.)
- Triaged changed from No to Yes
- Status changed from NEW to ASSIGNED
- Assignee set to ggainey
@daviddavis - leaving it unset opens pulp up to an easy DOS of "call a POST api with a Really Big Body and run Pulp out of memory". I'm proposing we just double the django default to 512Mb - that's still a huge amount of data.
As noted in the bugzilla, given the way Django settings work in Pulp-2, a user could override by changing the default in Django's global_settings.py, or by changing webservices/settings.py, and restarting. Neither of those are perfect, because those files will be overridden when updating Pulp - but they're as good as it gets, for Pulp-2.
ggainey you said 512Mb but I think you mean 5MB? That sounds good to me.
daviddavis wrote:
ggainey you said 512Mb but I think you mean 5MB? That sounds good to me.
Gah - aye, 5Mb. 512Mb is, perhaps, overkill :)
- Status changed from ASSIGNED to POST
- Status changed from POST to MODIFIED
- Platform Release set to 2.21.0
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
We are removing the 'API' category per open floor discussion June 16, 2020.
Also available in: Atom
PDF
.
Increase memory-limit for incoming API requests
closes #4214 https://pulp.plan.io/issues/4214