Pulp

I suggest to include a pointer to what should be removed ;-) I think we are talking about https://github.com/pulp/pulpcore/blob/4a55a4411db1a5904716233852b5837e5f7a01aa/pulpcore/app/models/progress.py#L139-L142, right?

My 2 cents:

I agree that the implementation is implicit and somewhat surprising, but I still find the functionality very useful.

There are stages that work best in batches, but there are also stages that don't. The downloader is one example; parsing meta-data from the remote in the first stage is another. As a plugin writer, I wouldn't like to be forced to work in arbitrary batches just to optimize reporting. Also, for some workloads like downloading, the time based approach fits better than an approach based on batches (what batch size to choose to get decent reporting?).

My proposal is to go from implicit to explicit:

• save() just saves the object (and possibly records a timestamp needed by other methods)
• update_progress(increment) increments by the given increment and saves the object (hence the update in the name)
• lazy_update_progress(increment) increments by the given number and may save using time-based throttling (the interval could be made explicit by adding it as a default parameter to the constructor).
• remove iter() and increment() (there is no use case to increment by 1 and save)
• possibly add an iterator for batches (would wrap the batch iterator in stages). It saves after every batch.