Migration Plugin

Story #3821

Updated by dkliban@redhat.com almost 5 years ago

h1. Description 

 Pulp 3 will provide optional plugins that will enable users to migrate their Pulp 2 instances to Pulp 3. The main package will be called pulp-2to3-migrate. Each plugin will also provide a package that will enable migration of the content types managed by that plugin.  

 After installing pulp-2to3-migrate on a Pulp 3 system, users will be able to use the REST API on Pulp 3 to start the migration of the Pulp 2 instance to Pulp 3. Users will be able to describe the final state of their Pulp 3 instance using a Migration Plan (MP). The MP is a JSON document that describes which artifacts, content, repositories, repository versions, publication, remotes, and distributions get created in Pulp 3 from their equivalent counterparts in Pulp 2. Users will be able to use Pulp 2's REST API    to generate a default MP for migrating everything. Users will then submit the MP via REST API to Pulp 3. 

 After the migration task start running in Pulp 3, the user will be able to use the REST API to query the mappings between Pulp 2 and Pulp 3 resources. 

 h1. Features 

 h3. Pulp 2 will provide a REST API that returns a default Migration Plan.  

 This Migration Plan will migrate all content, all repositories, all importers, and all distributors. 

 h3. Pulp 3 will provide a REST API that will accept three parameters: Migration Plan, concurrency, and dry_run. 

 This API will dispatch one more tasks to perform the migration. The default concurrency will be (Pulp 3 worker count) - 1. The concurrency will determine the number of tasks that will be dispatched in parallel. The tasking system will limit one migration "operation" at a time, although many tasks may run concurrently for that operation. 

 This API will dispatch one more tasks to perform the migration. The default concurrency will be (Pulp 3 worker count) - 1. The concurrency will determine the number of tasks that will be dispatched in parallel. The tasking system will limit one migration "operation" at a time, although many tasks may run concurrently for that operation. 

 When dry_run is set to True, the Migration Plan validated and some concreate information about the plan is returned to the user. simply validated. Some validation that will be performed will include: 
 

 * JSON syntax 
 * do the listed Pulp 2 entities exist 
 If all validation passes, the user receives a list of Content types and Repositories from Pulp 2 that will be used to create content and repositories in Pulp 3. 

 h3. Pulp 3 will provide multiple REST APIs for querying mappings between content in Pulp 2 and Pulp 3.  

 *TODO:* figure out what exact APIs are needed  

 h1. Migration Plan schema 

 The Migration Plan (MP) will consist of 2 sections: Content and Repositories. The Content section of the MP will describe the state of Content units and their Artifacts in Pulp 3. The Repositories section will describe Repositories, RepositoryVersions, Publications, and Distributions. 

 h3. Examples 

 The MP below will:  

 * migrate all content. 
 * create a repository with name 'foo' in Pulp 3. 
 * create one repository version that will have the same content set as the 'my-repo-id' repo in Pulp 2 
 * implicitly create a Publication from this repository version\ 
 * create 2 distributions based on details of distributors in Pulp 2 
 * both of these distributions will serve the same publication.  

 <pre> 
 { 
     "content" : {}, 
     "repositories" : [ 
        { 
            "name" : "foo", 
            "repository_versions" : [ 
                 { 
                     "pulp2_repository" : "my-repo-id", 
                     "distributions":    [ 
                         {"pulp2_distributor" :    "pulp2-distributor-id"}, 
                         {"pulp2_distributor" :    "another-pulp2-distributor-id"} 
                     ] 
                } 
            ] 
         } 
   ] 
 } 
 </pre> 

 The MP below will: 

 * implicitly create artifacts and content units in Pulp 3 for all the content units in Pulp 2 for which a Pulp 3 plugin is installed 
 * implicitly create a repository with a single repository version in Pulp 3 for every repository in Pulp 2 
 * the content set of the repository version will match the content of the repository in Pulp 2 
 * implicitly create a Publication for all repositories that need publications 
 * implicitly create a Remote for every importer associated with the repository 
 * implicitly create a Distribution for every non-export distributor associated with the repository and associate either the RepositoryVersion or Publication with that Distribution 

 <pre> 
 { 
     "content" : {}, 
     "repositories" : [] 
 } 
 </pre> 

Back