Project

Profile

Help

Refactor #4346

Simplify the plugin api for writing stages

Added by mdellweg almost 2 years ago. Updated about 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Problem

With the invent of content with futures, that are able to feed back in earlier stages of the pipeline, the need came up for some sort of flow control in the batching of content process to prevent deadlocks [0].

To keep full authority over any future implementation of this flow control, i suggest to refactor the stages api with a simple but strict definition, what the implementation of a stage must do / can rely on.

Solution

  1. A stage must override async def run(self):.
  2. In run, items can be retrieved through either self.items or self.batches iterator.
  3. ..., items must be forwarded by self.put.

Further discussion can be found here [1].

[0] https://pulp.plan.io/issues/4296
[1] https://github.com/pulp/pulpcore-plugin/pull/35

Associated revisions

History

#1 Updated by gmbnomis almost 2 years ago

+1 to the alternative using run; one thing less that stage writers may forget (calling finish)

#2 Updated by bmbouter almost 2 years ago

+1 also to the run approach.

It looks like that is the current code pushed to this PR https://github.com/pulp/pulpcore-plugin/pull/35/files is that right? Does that mean https://github.com/pulp/pulpcore-plugin/pull/35/files is ready for review?

#3 Updated by mdellweg almost 2 years ago

Yes, that is the actual version (i just could not wait.). Ready for review.

#4 Updated by daviddavis almost 2 years ago

  • Tags Pulp 3, Pulp 3 RC Blocker added

#5 Updated by bmbouter almost 2 years ago

  • Status changed from NEW to POST
  • Assignee set to mdellweg

#6 Updated by bmbouter almost 2 years ago

  • Description updated (diff)
  • Status changed from POST to MODIFIED

All the work for this is done so I'm moving to MODIFIED and rewriting with its final solution

#7 Updated by daviddavis over 1 year ago

  • Sprint/Milestone set to 3.0.0

#8 Updated by bmbouter over 1 year ago

  • Tags deleted (Pulp 3, Pulp 3 RC Blocker)

#9 Updated by bmbouter about 1 year ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF