Refactor #4346
Updated by bmbouter almost 6 years ago
h2. Problem With the invent of content with futures, that are able to feed back in earlier stages of the pipeline, the need came up for some sort of flow control in the batching of content process to prevent deadlocks [0]. To keep full authority over any future implementation of this flow control, i suggest to refactor the stages api with a simple but strict definition, what the implementation of a stage must do / can rely on. h2. Solution The current proposal is: # A stage must override @async def __call__(self):@. # In @__call__@, items can be retrieved through either @self.items@ or @self.batches@ iterator. # ..., items must be forwarded by @self.put@. # A stage must signal that it is finished by calling @self.finish@. Alternatively: # A stage must override @async def run(self):@. # In @run@, items can be retrieved through either @self.items@ or @self.batches@ iterator. # ..., items must be forwarded by @self.put@. Further discussion can be found here [1]. [0] https://pulp.plan.io/issues/4296 [1] https://github.com/pulp/pulpcore-plugin/pull/35
.