Project

Profile

Help

Story #9148

closed

Task #9256: [EPIC] Use shared task resources in plugin code

As a plugin writer i want to be able to share task resources for reading

Added by mdellweg over 3 years ago. Updated over 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Performance
Sprint:
Sprint 103
Quarter:

Description

The tasking system should be able to allow resources to be shared between tasks if only read access is requested.

The following design should allow to implement this without a migration and in a backwards compatible way:

  • reuse the reserved_resources_record ListField shared resources will be prefixed by "shared:" exclusive resources will be used as is
  • dispatch will add another parameter "shared_resources" in a backwards compatible way
  • workers will augment the list of in_use_resources with a flag if any previous task requested a resource exclusive

Note: shared access that was scheduled after exclusive access still needs to wait. That is, one task requesting exclusive access to a resource will introduce a barrier for all other tasks requesting the same resource in any way.

https://hackmd.io/Y0l9nBm9SFmPiIBXBz8lwQ?view#Idea-Example-with-shared-and-exclusive-locks


Related issues

Related to RPM Support - Story #9255: As a user, copy tasks which have the same source repos are run in parallelCLOSED - CURRENTRELEASEdalley

Actions
Actions #1

Updated by ipanova@redhat.com over 3 years ago

  • Description updated (diff)
Actions #3

Updated by mdellweg over 3 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to mdellweg
Actions #4

Updated by pulpbot over 3 years ago

  • Status changed from ASSIGNED to POST
Actions #5

Updated by dalley over 3 years ago

  • Sprint/Milestone set to 3.15.0
  • Sprint set to Sprint 103

Added by mdellweg over 3 years ago

Revision 66744406 | View on GitHub

Add shared_resources to task dispatch

This allows tasks that use overlapping resources read only to run concurrently. A setting ALLOW_SHARED_TASK_RESOURCES has been added to switch it on.

fixes #9148

Actions #7

Updated by mdellweg over 3 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #8

Updated by ttereshc over 3 years ago

  • Related to Story #9255: As a user, copy tasks which have the same source repos are run in parallel added
Actions #9

Updated by mdellweg over 3 years ago

  • Parent issue set to #9256
Actions #10

Updated by pulpbot over 3 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF