Story #3209
Updated by amacdona@redhat.com almost 7 years ago
This is the parent task for all stories/tasks related to adding RepositoryVersions. h2. Repository Version CRUD h3. CREATE: New Repository Version can be created in 2 ways, (1) directly with the Repository Versions Viewset (3234) and (2) action endpoints defined by each plugin (see "actions" below). Manual add/remove is done by a POST to /v3/repositories/<repo_id>/versions/. The user specifies the content to add and the content to remove in the POST body. The RepositoryVersions Viewset will deploy the task that creates a RepositoryVersion. Stories: https://pulp.plan.io/issues/3234 Create a new Repository Version /v3/repositories/<repo_uuid>/versions/ POST h3. LIST: list (v3/repositories/<repo_uuid>/versions/) and read (v3/repositories/<repo_uuid>/versions/<number>/) is handled by the Repository Version serializer and provides content href, added href, and removed href. Stories: https://pulp.plan.io/issues/3218 Add detail routes to viewset and serializer /v3/repositories/<repo_uuid>/versions/<number>/ should include links to detail routes, including: * content href * added content href * removed content href https://pulp.plan.io/issues/3235 On Repositories Serializer, return href for latest version https://pulp.plan.io/issues/3238 Filters for RepositoryVersions Bugs: https://pulp.plan.io/issues/3230 - bugfix - v3/repositories/<deleted_repo>/versions/ should return 404 h3. UPDATE: disabled RepositoryVersions are immutable, so the Viewset should not inherrit from the update mixin, so PUT and PATCH requests return a 405 (Method not allowed) h3. DELETE: All versions are able to be deleted. The latest version is simply deleted and the latest version becomes the previous latest. When a version other than latest is deleted, the changes made by that version are "squashed" into the following version. For example, Repository Av1 contains 2 packages (a and b). Av2 Adds version package c. Av3 removes package a. Av4 removes package b. Av1 (a, b) Av2 (a, b, c) [+c] Av3 (b, c) [-a] Av4 (c) [-b] If Av2 is deleted, Av3 now also adds "c". Av3(b, c) [+c, -a] If Av4 is deleted, A.latest() become Av3 and contains (b, c) When a user deletes a repository version, the content contained by all following versions does not change, but the content added and removed in the following version changes. When the latest version is deleted, Repository.latest rolls back to the previous version. Stories: https://pulp.plan.io/issues/3219 Delete Repository Versions: RepositoryVersion.latest is deleted RepositoryVersion that is not latest is "squashed" Bugs https://pulp.plan.io/issues/3233 Squash bug h2. Actions that create a Repository Version (including sync) If Pulp performs an action to create a Repository Version (as opposed to a manual add/remove), all of the logic is owned by each plugin. Interactions with the pulpcore database (RepositoryVersion, Repository, RepositoryContent) are handled by pulpcore utilities that are part of the plugin API. The utilities will provide a working directory, ensure that writes happen in transactions, clean up pulpcore database after failures and crashes. This change alters the plugin API. "sync", "publish" and other actions will be tasks that are written by each plugin (instead of a function on the model). The tasks are deployed by Plugin Viewsets which create action endpoints in the REST API. Transactions and Cleanup stories: https://pulp.plan.io/issues/3222 Add complete flag, and mark complete when action is finished. https://pulp.plan.io/issues/3226 Cleanup incomplete Repository Versions after a worker crash https://pulp.plan.io/issues/3225 Update Changeset with new arguments (repo_version, not old_version and new_version) Plugin API changes: https://pulp.plan.io/issues/3074 Remove sync code from pulpcore. Also acts as parent task for Plugin Stories: https://pulp.plan.io/issues/3285 Provide context manager to handle RepositoryVersion creation, finalization, and cleanup. Plugin Stories https://pulp.plan.io/issues/3260- File plugin sync https://pulp.plan.io/issues/3243 - Example plugin sync https://pulp.plan.io/issues/3294 - Python plugin sync h2. Actions that create a Publication (including publish) Actions that create Publications are handled similarly to actions the create RepositoryVersions. All of the logic is owned by each plugin. Interactions with the pulpcore database (Publication, Repository) are handled by pulpcore utilities that are part of the plugin API. Stories: https://pulp.plan.io/issues/3221 Move publish code from model to task https://pulp.plan.io/issues/3295 Write util to assist with publish (context manager?) Plugin stories: https://pulp.plan.io/issues/3296 file publish story https://pulp.plan.io/issues/3297 example publish story h2. Docs for Plugin Writer: https://pulp.plan.io/issues/3220 Plugin writer HOWTO guide to implement sync (and copy) https://pulp.plan.io/issues/3298 Plugin writer HOWTO guide to implement publish h2. Plugin Docs: https://pulp.plan.io/issues/3249 Update file plugin docs for repository versions https://pulp.plan.io/issues/3299 Update example plugin docs for repository versions https://pulp.plan.io/issues/33 Update python plugin docs for repository versions h2. Why this design? h3. Motivation The ContentUnits added to a repository change over time, but Pulp only stores the current state. This prevents a lot of great things like rolling back to a particluar snapshot, or understanding exactly what content is in place at any given time. h3. h2. Solution Introducing the Repository Version, which represents the ContentUnits in a repository after any given operation. For example, each of the following operations will produce a repository version. * Sync with from an importer * Add a set of units to a repository * Remove a set of units from a repository Any repository version can be deleted. Publications are changed to publish a specific RepositoryVersion, not just the latest content in a given repo. h3. h2. API examples <pre> POST /api/v3/importers/{type}/{importer_id}/sync/ Use plugin to create a new RepositoryVersion. POST /api/v3/repositories/{repo_id}/versions/ # create the new RepositoryVersion. specify the importer here, and/or units to add, and/or units to remove GET /api/v3/repository/{repo_id}/versions/{number} # The resource for a single repo version GET /api/v3/repositories/{repo_id}/versions/<id>/content/ # paginated list of all content in particular version GET /api/v3/repositories/{repo_id}/versions/<id>/added_content/ # paginated list of content added when a particular version was created GET /api/v3/repositories/{repo_id}/versions/<id>/removed_content/ # paginated list of content removed when a particular version was created </pre> h2. Technical details The _RepositoryVersion_ table stores all repository versions for all repositories. A repository version is a set of content. *RepositoryVersion* * [pk] *id* - The primary key. * [fk] *content* - The associated repository. * *created* - When the repository version was created. * *number* - A positive integer that uniquely identifies a version of a specific repository. Each new version for a repository should have this field set to 1 + the most recent version. * *complete* - When true, the action that creates the RepositoryVersion is finished and the version is ready to be used. The _RepositoryContent_ table stores association between a repository version and its contained content. *RepositoryContent* * [pk] *id* - The primary key. * [fk] *content* - The associated content. * [fk] *repository* - The associated repository. * [fk] *version_added* - The RepositoryVersion which added the referenced Content. * [fk] *version_removed* - The RepositoryVersion which removed the referenced Content.
.