Project

Profile

Help

Issue #4214

closed

Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE

Added by daviddavis over 3 years ago. Updated about 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
2.21.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Easy Fix, Pulp 2
Sprint:
Quarter:

Description

Submit a large request (> 2.5 MB) to a pulp 2 endpoint and it'll raise the error "body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE". It looks like in this case Katello was posting a rather long list of content units to /pulp/api/v2/repositories/<repo id>/actions/associate/.

Looks like this is a django setting that is by default 2.5 MB.

https://docs.djangoproject.com/en/2.1/ref/settings/#data-upload-max-memory-size

Actions #1

Updated by daviddavis over 3 years ago

  • Tags Easy Fix added
Actions #2

Updated by amacdona@redhat.com over 3 years ago

  • Is duplicate of Story #4196: As a user, I can upload files in chunks. added
Actions #3

Updated by amacdona@redhat.com over 3 years ago

  • Status changed from NEW to CLOSED - DUPLICATE

Increasing the max size is a workaround, the more important feature is to chunk large uploads. Not related to uploads

Actions #4

Updated by daviddavis over 3 years ago

  • Description updated (diff)
  • Status changed from CLOSED - DUPLICATE to NEW

It wasn't an upload. I've updated the description to add an example of a request.

Actions #5

Updated by daviddavis over 3 years ago

I think we should either disable DATA_UPLOAD_MAX_MEMORY_SIZE or bump up the size. 2.5 MB seems small especially since Katello will often enumerate a list of ids when performing actions like repo copy.

Actions #6

Updated by amacdona@redhat.com over 3 years ago

Maybe we should consider a new config option?

Actions #7

Updated by amacdona@redhat.com over 3 years ago

  • Is duplicate of deleted (Story #4196: As a user, I can upload files in chunks.)
Actions #8

Updated by CodeHeeler over 3 years ago

  • Triaged changed from No to Yes
Actions #9

Updated by bmbouter over 3 years ago

  • Tags Pulp 2 added
Actions #10

Updated by ggainey about 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ggainey
Actions #11

Updated by ggainey about 3 years ago

@daviddavis - leaving it unset opens pulp up to an easy DOS of "call a POST api with a Really Big Body and run Pulp out of memory". I'm proposing we just double the django default to 512Mb - that's still a huge amount of data.

As noted in the bugzilla, given the way Django settings work in Pulp-2, a user could override by changing the default in Django's global_settings.py, or by changing webservices/settings.py, and restarting. Neither of those are perfect, because those files will be overridden when updating Pulp - but they're as good as it gets, for Pulp-2.

Actions #12

Updated by daviddavis about 3 years ago

ggainey you said 512Mb but I think you mean 5MB? That sounds good to me.

Actions #13

Updated by ggainey about 3 years ago

daviddavis wrote:

ggainey you said 512Mb but I think you mean 5MB? That sounds good to me.

Gah - aye, 5Mb. 512Mb is, perhaps, overkill :)

Actions #14

Updated by ggainey about 3 years ago

  • Status changed from ASSIGNED to POST

Added by ggainey about 3 years ago

Revision cf9690c2

Increase memory-limit for incoming API requests

closes #4214 https://pulp.plan.io/issues/4214

Actions #15

Updated by ggainey about 3 years ago

  • Status changed from POST to MODIFIED
Actions #16

Updated by dalley almost 3 years ago

  • Platform Release set to 2.21.0
Actions #17

Updated by dalley almost 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #18

Updated by bmbouter about 2 years ago

  • Category deleted (14)

We are removing the 'API' category per open floor discussion June 16, 2020.

Also available in: Atom PDF