Project

Profile

Help

Story #5096

closed

[epic] As a user, I can export the content of a RepositoryVersion from one Pulp3 system and import on an air gapped Pulp3 system

Added by bmbouter over 4 years ago. Updated almost 4 years ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
Yes
Sprint Candidate:
No
Tags:
Katello
Sprint:
Quarter:

Description

Problem

The Pulp3-to-Pulp3 sync mechanisms are typically one Pulp syncing from another Pulp over the network. When the two Pulp3 systems are air gapped this sync mechanism won't work.

Plugins could make individual import/export functions, but can pulpcore do it once for all Content types?

Use Case: Full Export+Import

As a Pulp3 user I can export all of the content from one RepositoryVersion to a disk. Then walk that disk into the air gapped environment and mount it. Add the RepositoryVersion to an air gapped Pulp as part of a specified, existing Repository.

Use Case: Incremental Export+Import

As a Pulp3 user who has exported a RepositoryVersion earlier, I can specify that RepositoryVersion and it will export the repository containing all content differences between that RepositoryVersion and the latest version of that Repository.

Challenges

Artifact and ContentArtifact objects

The Content units created need to also create corresponding Artifact and ContentArtifact objects. Artifacts already existing on the import system should not be disturbed but used.

Foreign Keys to/from Content Model

Content units sometimes have other non-Content models that are related to them, e.g. in pulp_rpm the UpdateRecord has related UpdateCollection and UpdateCollectionPackage records. These may or may not need to be duplicated and likely need plugin writers to be involved.

Solution

Export

Make an export API that takes a 'repository_version' and it will generate a task that will create the export archive for the user. All Content types in the repository version, their Artifacts, and ContentArtifact objects are exported with django-import-export. This will make the file available for downloading via the pulp-content app somehow as a single archive.

Additionally the Export takes a 'prev_export_repository_version' option which will be used as the diff for all content between 'prev_export_repository_version' and 'repository_version'.

Import

Using the upload API the tarball can be uploaded in parallel efficiently. An import API that takes a 'repository' and it will generate a task that will import the archive for the user.

Technical Plan

To use django-import-export, plugin writers would provide a customized resource for their Content model and this would be put into a resources.py and imported by the Django plugin loader causing it to import for each plugin.

If the plugin writer doesn't provide one and the generic one will work for them, they do nothing, and pulpcore will create a generic one at Runtime

pulpcore would also provide Resources for Artifact and ContentArtifact which the import task would also handle.

Open Questions

How will users at import time specify if they want dry-run or not?


Related issues

Related to Pulp - Story #6134: [EPIC] Pulp import/exportCLOSED - CURRENTRELEASE

Actions
Has duplicate Ansible Plugin - Story #5088: support ansible collection content exportCLOSED - DUPLICATE

Actions
Actions #1

Updated by bmbouter over 4 years ago

  • Description updated (diff)
Actions #2

Updated by bmbouter over 4 years ago

  • Description updated (diff)
Actions #3

Updated by daviddavis over 4 years ago

  • Groomed changed from No to Yes
  • Sprint Candidate changed from No to Yes
Actions #4

Updated by rchan over 4 years ago

  • Sprint set to Sprint 56
Actions #5

Updated by bmbouter over 4 years ago

  • Has duplicate Story #5088: support ansible collection content export added
Actions #6

Updated by bmbouter over 4 years ago

  • Tags Katello-P2 added

This duplicates a Katello P2 ( https://pulp.plan.io/issues/5088. ), so this should receive the Katello P2 label.

Actions #7

Updated by rchan over 4 years ago

  • Sprint changed from Sprint 56 to Sprint 57
Actions #8

Updated by rchan over 4 years ago

  • Sprint changed from Sprint 57 to Sprint 58
Actions #9

Updated by rchan over 4 years ago

  • Sprint changed from Sprint 58 to Sprint 59
Actions #10

Updated by daviddavis over 4 years ago

Removing from the sprint since this is not a 3.0 blocker.

Actions #11

Updated by daviddavis over 4 years ago

  • Sprint deleted (Sprint 59)
Actions #12

Updated by daviddavis over 4 years ago

  • Tags Pulp 2 to 3 Migrate added
  • Tags deleted (Katello-P2)
Actions #13

Updated by rchan about 4 years ago

  • Sprint Candidate deleted (Yes)
Actions #14

Updated by jsherril@redhat.com about 4 years ago

  • Sprint Candidate set to No
  • Tags Katello-P2 added
Actions #15

Updated by daviddavis about 4 years ago

  • Related to Story #6134: [EPIC] Pulp import/export added
Actions #16

Updated by daviddavis about 4 years ago

Superseded by #6134

Actions #17

Updated by bmbouter about 4 years ago

  • Status changed from NEW to CLOSED - DUPLICATE

closing as a duplicate of https://pulp.plan.io/issues/6134

Actions #18

Updated by ggainey almost 4 years ago

  • Tags Katello added
  • Tags deleted (Katello-P2)
Actions #19

Updated by bmbouter almost 4 years ago

  • Tags deleted (Pulp 2 to 3 Migrate)

We're removing the 'Pulp 2 to 3 Migrate' tag to allow membership of the Migration Plugin project in Redmine to be the controlling area https://pulp.plan.io/projects/migration

Also available in: Atom PDF