Project

Profile

Help

Issue #4214

Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE

Added by daviddavis 12 months ago. Updated about 2 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
API
Sprint/Milestone:
-
Start date:
Due date:
Severity:
2. Medium
Version:
Platform Release:
2.21.0
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Easy Fix, Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

Submit a large request (> 2.5 MB) to a pulp 2 endpoint and it'll raise the error "body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE". It looks like in this case Katello was posting a rather long list of content units to /pulp/api/v2/repositories/<repo id>/actions/associate/.

Looks like this is a django setting that is by default 2.5 MB.

https://docs.djangoproject.com/en/2.1/ref/settings/#data-upload-max-memory-size

Associated revisions

Revision cf9690c2 View on GitHub
Added by ggainey 4 months ago

Increase memory-limit for incoming API requests

closes #4214
https://pulp.plan.io/issues/4214

History

#1 Updated by daviddavis 12 months ago

  • Tags Easy Fix added

#2 Updated by amacdona@redhat.com 12 months ago

  • Duplicates Story #4196: As a user, I can upload files in chunks. added

#3 Updated by amacdona@redhat.com 12 months ago

  • Status changed from NEW to CLOSED - DUPLICATE

Increasing the max size is a workaround, the more important feature is to chunk large uploads. Not related to uploads

#4 Updated by daviddavis 12 months ago

  • Description updated (diff)
  • Status changed from CLOSED - DUPLICATE to NEW

It wasn't an upload. I've updated the description to add an example of a request.

#5 Updated by daviddavis 12 months ago

I think we should either disable DATA_UPLOAD_MAX_MEMORY_SIZE or bump up the size. 2.5 MB seems small especially since Katello will often enumerate a list of ids when performing actions like repo copy.

#6 Updated by amacdona@redhat.com 12 months ago

Maybe we should consider a new config option?

#7 Updated by amacdona@redhat.com 12 months ago

  • Duplicates deleted (Story #4196: As a user, I can upload files in chunks.)

#8 Updated by CodeHeeler 12 months ago

  • Triaged changed from No to Yes

#9 Updated by bmbouter 7 months ago

  • Tags Pulp 2 added

#10 Updated by ggainey 4 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ggainey

#11 Updated by ggainey 4 months ago

@daviddavis - leaving it unset opens pulp up to an easy DOS of "call a POST api with a Really Big Body and run Pulp out of memory". I'm proposing we just double the django default to 512Mb - that's still a huge amount of data.

As noted in the bugzilla, given the way Django settings work in Pulp-2, a user could override by changing the default in Django's global_settings.py, or by changing webservices/settings.py, and restarting. Neither of those are perfect, because those files will be overridden when updating Pulp - but they're as good as it gets, for Pulp-2.

#12 Updated by daviddavis 4 months ago

@ggainey you said 512Mb but I think you mean 5MB? That sounds good to me.

#13 Updated by ggainey 4 months ago

daviddavis wrote:

@ggainey you said 512Mb but I think you mean 5MB? That sounds good to me.

Gah - aye, 5Mb. 512Mb is, perhaps, overkill :)

#14 Updated by ggainey 4 months ago

  • Status changed from ASSIGNED to POST

#15 Updated by ggainey 4 months ago

  • Status changed from POST to MODIFIED

#16 Updated by dalley 3 months ago

  • Platform Release set to 2.21.0

#17 Updated by dalley about 2 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF