Pulp

Problem¶

With the invent of content with futures, that are able to feed back in earlier stages of the pipeline, the need came up for some sort of flow control in the batching of content process to prevent deadlocks [0].

To keep full authority over any future implementation of this flow control, i suggest to refactor the stages api with a simple but strict definition, what the implementation of a stage must do / can rely on.

Solution¶

1. A stage must override async def run(self):.
2. In run, items can be retrieved through either self.items or self.batches iterator.
3. ..., items must be forwarded by self.put.

Further discussion can be found here [1].

[0] https://pulp.plan.io/issues/4296
[1] https://github.com/pulp/pulpcore-plugin/pull/35