Project

Profile

Help

Refactor #4346

Simplify the plugin api for writing stages

Added by mdellweg about 2 years ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Problem

With the invent of content with futures, that are able to feed back in earlier stages of the pipeline, the need came up for some sort of flow control in the batching of content process to prevent deadlocks [0].

To keep full authority over any future implementation of this flow control, i suggest to refactor the stages api with a simple but strict definition, what the implementation of a stage must do / can rely on.

Solution

  1. A stage must override async def run(self):.
  2. In run, items can be retrieved through either self.items or self.batches iterator.
  3. ..., items must be forwarded by self.put.

Further discussion can be found here [1].

[0] https://pulp.plan.io/issues/4296
[1] https://github.com/pulp/pulpcore-plugin/pull/35

Associated revisions

Revision 5ed0431d View on GitHub
Added by dalley about 2 years ago

Adapt to changes in the stages API

re #4346 https://pulp.plan.io/issues/4346

History

#1 Updated by gmbnomis about 2 years ago

+1 to the alternative using run; one thing less that stage writers may forget (calling finish)

#2 Updated by bmbouter about 2 years ago

+1 also to the run approach.

It looks like that is the current code pushed to this PR https://github.com/pulp/pulpcore-plugin/pull/35/files is that right? Does that mean https://github.com/pulp/pulpcore-plugin/pull/35/files is ready for review?

#3 Updated by mdellweg about 2 years ago

Yes, that is the actual version (i just could not wait.). Ready for review.

#4 Updated by daviddavis about 2 years ago

  • Tags Pulp 3, Pulp 3 RC Blocker added

#5 Updated by bmbouter about 2 years ago

  • Status changed from NEW to POST
  • Assignee set to mdellweg

#6 Updated by bmbouter about 2 years ago

  • Description updated (diff)
  • Status changed from POST to MODIFIED

All the work for this is done so I'm moving to MODIFIED and rewriting with its final solution

#7 Updated by daviddavis almost 2 years ago

  • Sprint/Milestone set to 3.0.0

#8 Updated by bmbouter almost 2 years ago

  • Tags deleted (Pulp 3, Pulp 3 RC Blocker)

#9 Updated by bmbouter over 1 year ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF