Project

Profile

Help

Issue #9266

repository 'modify' touches all the content being added one by one

Added by dkliban@redhat.com 2 months ago. Updated 22 days ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 105
Quarter:

Description

Bulk touch feature of the BulkQuerySet should be used to reduce number of queries that are executed in the viewset handling the Repository Modify API.

Removing the iteration of the content in the viewset will require the task to determine if any of the content to be added doesn't exist. The task needs to raise a RuntimeError with a list of pulp_href's for the content that doesn't exist.


Related issues

Related to Pulp - Issue #9243: 30-50% re-sync performance regression due to touch() of content and artifacts during syncCLOSED - CURRENTRELEASE<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Copied to Pulp - Backport #9401: Backport #9266 "repository 'modify' touches all the content being added one by one" to 3.14.zCLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 4023fce7 View on GitHub
Added by Lubos Mjachky about 1 month ago

Do bulk_touch instead of touching one by one

closes #9266

History

#1 Updated by dkliban@redhat.com 2 months ago

  • Related to Issue #9243: 30-50% re-sync performance regression due to touch() of content and artifacts during sync added

#2 Updated by dkliban@redhat.com about 2 months ago

  • Triaged changed from No to Yes
  • Sprint set to Sprint 104

#3 Updated by dalley about 2 months ago

Additionally, this line is problematic for the same reason, one query per content unit. The same issue led to https://pulp.plan.io/issues/7483 for the RPM plugin

https://github.com/pulp/pulpcore/blob/de67d41c39d9d4bde2e8f86e4bf3f51d6cc774e7/pulpcore/plugin/actions.py#L47

There was some previous discussion about this on Matrix covering some of the competing requirements: https://riot.im/app/#/room/#pulp-dev:matrix.org/$8_vHR2IhnUvnjSU5XeimYEKXov-ed9u_Us0YtMzqvPE

#4 Updated by rchan about 2 months ago

  • Sprint changed from Sprint 104 to Sprint 105

#5 Updated by dalley about 2 months ago

  • Priority changed from Normal to High
  • Severity changed from 2. Medium to 3. High

Bumping the prio because with a lot of content in the operation, it can easily time out the whole request. We don't have any tests running at that scale but Katello is running into this issue.

#6 Updated by lmjachky about 1 month ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to lmjachky

#7 Updated by dalley about 1 month ago

  • Sprint/Milestone set to 3.16.0

#8 Updated by dalley about 1 month ago

  • Copied to Backport #9401: Backport #9266 "repository 'modify' touches all the content being added one by one" to 3.14.z added

#9 Updated by pulpbot about 1 month ago

  • Status changed from ASSIGNED to POST

#10 Updated by Anonymous about 1 month ago

  • Status changed from POST to MODIFIED

#12 Updated by pulpbot 22 days ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF