Migration Plugin

Story #3821

Updated by ttereshc about 5 years ago

h1. Description 

 Pulp 3 will provide a plugin that will enable users to migrate their Pulp 2 instances to Pulp 3. The package will be called pulp-2to3-migration. Plugin writers will contribute to it to provide support of migration of the content types managed by that plugin.  

 After installing pulp-2to3-migration on a Pulp 3 system, users will be able to use the REST API on Pulp 3 to start the migration of the Pulp 2 instance to Pulp 3. Users will be able to describe the final state of their Pulp 3 instance for a specific plugin using a Migration Plan (MP). The MP is a JSON document that describes which artifacts, content, repositories, repository versions, publication, remotes, and distributions get created in Pulp 3 from their equivalent counterparts in Pulp 2. Users will be able to use Pulp 2's REST API    to generate a default MP for migrating everything. Users will then submit the MP via REST API to Pulp 3. 

 After the migration task start running in Pulp 3, the user will be able to use the REST API to query the mappings between Pulp 2 and Pulp 3 resources. 

 h1. Features 

 h3. Pulp 2 will provide a REST API that returns a default Migration Plan.  

 This Migration Plan will migrate all content, all repositories, all importers, and all distributors. 

 h3. Pulp 3 will provide a REST API that: 

  
  - creates a Migration Plan 
  - executes a Migration plan in one of 3 modes (run the migration without validation, run migration if MP is validated, run validation only - dry-run). 

 This API will dispatch one or more tasks to perform the migration. The tasking system will limit one migration "operation" at a time, although many tasks may run concurrently for that operation. 

 When dry_run is set to True, the Migration Plan validated and some information about the plan is returned to the user. Some validation that will be performed will include: 
 * JSON syntax 
 * do the listed Pulp 2 entities exist 
 -If all validation passes, the user receives a list of Content types and Repositories from Pulp 2 that will be used to create content and repositories in Pulp 3.- 

 h3. Pulp 3 will provide multiple REST APIs for querying mappings between Pulp 2 and Pulp 3.  

  
 One for content, one for repository and related resources    - importers, distributors. 


 h1. Migration Plan schema 

 The Migration Plan (MP) will consist of 2 sections: Content and Repositories. The Content section of the plugin type MP will describe the state of Content units and repositories specification for each plugin. If the desire is to migrate content only, repositories specification should not be mentioned at all.    their Artifacts in Pulp 3. The Repositories section will describe Repositories, RepositoryVersions, Publications, and Distributions to create. Distributions. 

 h3. Examples 

 The MP below will:  

 * migrate all content for ISO and Docker plugins content. 
 * create a repository with name 'foo' in Pulp 3. 
 * create a Remote in Pulp 3 based on details of an importer for the repo "my-repo-id-for-importer" in Pulp 2 
 * create one repository version that will have the same content set as the ISO 'my-repo-id' repo in Pulp 2 
 * implicitly create a Publication from this repository version version\ 
 * create 2 distributions based on details of distributors in Pulp 2 
 * both of these distributions will serve the same publication.  

 <pre> 
  
 { 
     "plugins": "content" : {}, 
     "repositories" : [ 
        {  
          "type": "iso",  
          "repositories" 
            "name" :    [ "foo", 
            { 
              "name": "foo", 
              "pulp2_importer_repository_id":    "my-repo-id-for-importer"  
              "repository_versions": "repository_versions" : [ 
                  
                 { 
                    "pulp2_repository_id": 
                     "pulp2_repository" : "my-repo-id", 
                    "distributor_ids": 
                     "distributions":    ["distributor_id1", "distributor_id2"] 
                   },           
                [ 
                         {"pulp2_distributor" :    "pulp2-distributor-id"}, 
                         {"pulp2_distributor" :    "another-pulp2-distributor-id"} 
                     ] 
             
                } 
            ] 
         ] 
      }, 
      { 
        "type": "docker", 
      } 
   ] 
 } 
 </pre> 

 The MP below will: 

 *migrate * implicitly create artifacts and content units in Pulp 3 for all ISO the content units in Pulp 2 for which a Pulp 3 plugin is installed 
 * implicitly create a repository with a single repository version in Pulp 3 for every repository in Pulp 2 
 * the content set of the repository version will match the content of the repository in Pulp 2 
 * implicitly create a Publication for all repositories that need publications 
 * implicitly create a Remote for every importer associated with the repository 
 * implicitly create a Distribution for every non-export distributor associated with the repository and associate either the RepositoryVersion or Publication with that Distribution 

 <pre> 
 { 
     "plugins": [ 
        {  
          "type": "iso", 
          "repositories": [], 
        } 
      ] 
 } 
 </pre> 

 The MP below will: 

 * migrate all downloaded ISO content 
 * not migrate any on_demand content 
 * not migrate/create any repositories/remotes/distributions 

 <pre> 
 { "content" : {}, 
     "plugins": [ 
        {  
          "type": "iso", 
        } 
      ] "repositories" : [] 
 } 
 </pre>

Back