Story #3209

Updated by over 4 years ago

This is the parent task for all stories/tasks related to adding RepositoryVersions. 

 h2. Repository Version CRUD 

 h3. CREATE: 

 New Repository Version can be created in 2 ways, (1) directly with the Repository Versions Viewset (3234) and (2) action endpoints defined by each plugin (see "actions" below). 

 Manual add/remove is done by a POST to /v3/repositories/<repo_id>/versions/. The user specifies the content to add and the content to remove in the POST body. The RepositoryVersions Viewset will deploy the task that creates a RepositoryVersion. 

 Stories: Create a new Repository Version 
 /v3/repositories/<repo_uuid>/versions/ POST 

 h3. LIST: 

 list (v3/repositories/<repo_uuid>/versions/) and read (v3/repositories/<repo_uuid>/versions/<number>/) is handled by the Repository Version serializer and provides content href, added href, and removed href. 
 Stories: Add detail routes to viewset and serializer 
     /v3/repositories/<repo_uuid>/versions/<number>/ should include links to detail routes, including: 
 * content href 
 * added content href 
 * removed content href On Repositories Serializer, return href for latest version Filters for RepositoryVersions 

 Bugs: - bugfix - v3/repositories/<deleted_repo>/versions/ should return 404 

 h3. UPDATE: disabled 

 RepositoryVersions are immutable, so the Viewset should not inherrit from the update mixin, so PUT and PATCH requests return a 405 (Method not allowed) 

 h3. DELETE: 

 All versions are able to be deleted. The latest version is simply deleted and the latest version becomes the previous latest. When a version other than latest is deleted, the changes made by that version are "squashed" into the following version. 

 For example, Repository Av1 contains 2 packages (a and b). Av2 Adds version package c. Av3 removes package a. Av4 removes package b. 
 Av1 (a, b) 
 Av2 (a, b, c) [+c]  
 Av3 (b, c) [-a] 
 Av4 (c) [-b] 

 If Av2 is deleted, Av3 now also adds "c". Av3(b, c) [+c, -a] 
 If Av4 is deleted, A.latest() become Av3 and contains (b, c) 

 When a user deletes a repository version, the content contained by all following versions does not change, but the content added and removed in the following version changes. 
 When the latest version is deleted, Repository.latest rolls back to the previous version. 

 Stories: Delete Repository Versions: 
         RepositoryVersion.latest is deleted 
         RepositoryVersion that is not latest is "squashed" 

 Bugs Squash bug 

 h2. Actions that create a Repository Version (including sync) 

 If Pulp performs an action to create a Repository Version (as opposed to a manual add/remove), all of the logic is owned by each plugin. Interactions with the pulpcore database (RepositoryVersion, Repository, RepositoryContent) are handled by pulpcore utilities that are part of the plugin API. The utilities will provide a working directory, ensure that writes happen in transactions, clean up pulpcore database after failures and crashes. 

 This change alters the plugin API. "sync", "publish" and other actions will be tasks that are written by each plugin (instead of a function on the model). The tasks are deployed by Plugin Viewsets which create action endpoints in the REST API. 

 Transactions and Cleanup stories: Add complete flag, and mark complete when action is finished. Cleanup incomplete Repository Versions after a worker crash Update Changeset with new arguments (repo_version, not old_version and new_version) 

 Plugin API changes: Remove sync code from pulpcore. Also acts as parent task for Plugin Stories: Provide context manager to handle RepositoryVersion creation, finalization, and cleanup. 

 Plugin Stories File plugin sync - Example plugin sync - Python plugin sync 

 h2. Actions that create a Publication (including publish) 

 Actions that create Publications are handled similarly to actions the create RepositoryVersions. All of the logic is owned by each plugin. Interactions with the pulpcore database (Publication, Repository) are handled by pulpcore utilities that are part of the plugin API.  

 Stories: Move publish code from model to task Write util to assist with publish (context manager?) 

 Plugin stories: file publish story example publish story 

 h2. Docs for Plugin Writer: Plugin writer HOWTO guide to implement sync (and copy) Plugin writer HOWTO guide to implement publish 
 h2. Plugin Docs: Update file plugin docs for repository versions Update example plugin docs for repository versions Update python plugin docs for repository versions 

 h2. Why this design? 

 h3. Motivation 

 The ContentUnits added to a repository change over time, but Pulp only stores the current state. This prevents a lot of great things like rolling back to a particluar snapshot, or understanding exactly what content is in place at any given time. 

 h3. Solution 

 Introducing the Repository Version, which represents the ContentUnits in a repository after any given operation. For example, each of the following operations will produce a repository version. 

 * Sync with an importer 
 * Add a set units to a repository 
 * Remove a set of units from a repository 

 Any repository version can be deleted. Publications are changed to publish a specific RepositoryVersion, not just the latest content in a given repo. 

 h3. API examples 

 POST /api/v3/importers/{type}/{importer_id}/sync/ Use plugin to create a new RepositoryVersion. 
 POST /api/v3/repositories/{repo_id}/versions/    # create the new RepositoryVersion. specify units to add, and/or units to remove 
 GET /api/v3/repository/{repo_id}/versions/{number}    # The resource for a single repo version 
 GET /api/v3/repositories/{repo_id}/versions/<id>/content/    # paginated list of all content in particular version 
 GET /api/v3/repositories/{repo_id}/versions/<id>/added_content/    # paginated list of content added when a particular version was created 
 GET /api/v3/repositories/{repo_id}/versions/<id>/removed_content/    # paginated list of content removed when a particular version was created 

 h2. Technical details 

 The _RepositoryVersion_ table stores all repository versions for all repositories. A repository version is a set of content. 

 * [pk] *id* - The primary key. 
 * [fk] *content* - The associated repository. 
 * *created* - When the repository version was created. 
 * *number* - A positive integer that uniquely identifies a version of a specific repository. Each new version for a repository should have this field set to 1 + the most recent version. 
 * *complete* - When true, the action that creates the RepositoryVersion is finished and the version is ready to be used. 

 The _RepositoryContent_ table stores association between a repository version and its contained content.  

 * [pk] *id* - The primary key. 
 * [fk] *content* - The associated content. 
 * [fk] *repository* - The associated repository. 
 * [fk] *version_added* - The RepositoryVersion which added the referenced Content. 
 * [fk] *version_removed* - The RepositoryVersion which removed the referenced Content.