Issue #9266: repository 'modify' touches all the content being added one by one - Pulp

Actions

Send by e-mail Copy link

Issue #9266

closed

repository 'modify' touches all the content being added one by one

Added by dkliban@redhat.com over 3 years ago. Updated about 3 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

High

Assignee:

lmjachky

Category:

Sprint/Milestone:

3.16.0

Start date:

Due date:

Estimated time:

Severity:

3. High

Version:

Platform Release:

OS:

Triaged:

Yes

Groomed:

Sprint Candidate:

Tags:

Sprint:

Sprint 105

Quarter:

Description

Bulk touch feature of the BulkQuerySet should be used to reduce number of queries that are executed in the viewset handling the Repository Modify API.

Removing the iteration of the content in the viewset will require the task to determine if any of the content to be added doesn't exist. The task needs to raise a RuntimeError with a list of pulp_href's for the content that doesn't exist.

Related issues

Actions

Copy link

Updated by dkliban@redhat.com over 3 years ago

Related to Issue #9243: 30-50% re-sync performance regression due to touch() of content and artifacts during sync added

Actions

Copy link

Updated by dkliban@redhat.com over 3 years ago

Triaged changed from No to Yes
Sprint set to Sprint 104

Actions

Copy link

Updated by dalley about 3 years ago

Additionally, this line is problematic for the same reason, one query per content unit. The same issue led to https://pulp.plan.io/issues/7483 for the RPM plugin

https://github.com/pulp/pulpcore/blob/de67d41c39d9d4bde2e8f86e4bf3f51d6cc774e7/pulpcore/plugin/actions.py#L47

There was some previous discussion about this on Matrix covering some of the competing requirements: https://riot.im/app/#/room/#pulp-dev:matrix.org/$8_vHR2IhnUvnjSU5XeimYEKXov-ed9u_Us0YtMzqvPE

Actions

Copy link

Updated by rchan about 3 years ago

Sprint changed from Sprint 104 to Sprint 105

Actions

Copy link

Updated by dalley about 3 years ago

Priority changed from Normal to High
Severity changed from 2. Medium to 3. High

Bumping the prio because with a lot of content in the operation, it can easily time out the whole request. We don't have any tests running at that scale but Katello is running into this issue.

Actions

Copy link