Project

Profile

Help

Story #6024

Updated by gmbnomis 11 months ago

h1. Motivation

Plugins need to iterate over the content in a repository version to create a publication. Pulp
should offer a batching iterator over content in order to limit the memory footprint and to allow
efficient use of content batches in DB operations.

h1. Solution

We could make @batch_qs()@ available to plugin writers. But this generator is hard to use correctly because of the ordering requirement (even in pulpcore it has not been used correctly).

Instead, add a generator to @RepositoryVersion@, which does "the right thing" by default and is customizable by plugin writers if necessary.

For an example use case, see https://github.com/gmbnomis/pulp_cookbook/blob/batched_publication_upstream/pulp_cookbook/app/tasks/publishing.py#L87-L117

Back