Project

Profile

Help

Issue #9655

closed

requesting lots of packages with a subset of fields still causes changelogs, filelists, requires, etc.. to be loaded, resulting in slow requests

Added by jsherril@redhat.com about 2 years ago. Updated about 2 years ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Katello, Performance
Sprint:
Quarter:

Description

Ticket moved to GitHub: "pulp/pulp_rpm/2312":https://github.com/pulp/pulp_rpm/issues/2312


When we make a request like this:

/pulp/api/v3/content/rpm/packages/?fields=pulp_href%2Cname%2Cversion%2Crelease%2Carch%2Cepoch%2Csummary%2Cis_modular%2Crpm_sourcerpm%2Clocation_href%2CpkgId&limit=1&offset=1000&repository_version=%2Fpulp%2Fapi%2Fv3%2Frepositories%2Frpm%2Frpm%2Fc6b93206-22bb-4c58-ba07-828c32326330%2Fversions%2F1%2F

where some subset of fields is requested, we're still seeing all of the fields being loaded from the database. The result is that the query takes much much longer than it should

Testing on the rhel7 repo, it took about ~100 seconds on my machine to fetch all 32K packages specifying the fields above. If I comment out the filelists, changelogs, provides/requires from the model and serializers, that drops down to ~33 seconds. This is a huge improvement.

Reading this: https://stackoverflow.com/questions/53319787/how-can-i-select-specific-fields-in-django-rest-framework

it looks like its may be possible to exclude certain fields from the query itself fairly easily.

Actions #1

Updated by fao89 about 2 years ago

  • Description updated (diff)
  • Status changed from NEW to CLOSED - DUPLICATE

Also available in: Atom PDF