Project

Profile

Help

Story #5096

[epic] As a user, I can export the content of a RepositoryVersion from one Pulp3 system and import on an air gapped Pulp3 system

Added by bmbouter over 1 year ago. Updated 5 months ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Category:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
Yes
Sprint Candidate:
No
Tags:
Katello
Sprint:
Quarter:

Description

Problem

The Pulp3-to-Pulp3 sync mechanisms are typically one Pulp syncing from another Pulp over the network. When the two Pulp3 systems are air gapped this sync mechanism won't work.

Plugins could make individual import/export functions, but can pulpcore do it once for all Content types?

Use Case: Full Export+Import

As a Pulp3 user I can export all of the content from one RepositoryVersion to a disk. Then walk that disk into the air gapped environment and mount it. Add the RepositoryVersion to an air gapped Pulp as part of a specified, existing Repository.

Use Case: Incremental Export+Import

As a Pulp3 user who has exported a RepositoryVersion earlier, I can specify that RepositoryVersion and it will export the repository containing all content differences between that RepositoryVersion and the latest version of that Repository.

Challenges

Artifact and ContentArtifact objects

The Content units created need to also create corresponding Artifact and ContentArtifact objects. Artifacts already existing on the import system should not be disturbed but used.

Foreign Keys to/from Content Model

Content units sometimes have other non-Content models that are related to them, e.g. in pulp_rpm the UpdateRecord has related UpdateCollection and UpdateCollectionPackage records. These may or may not need to be duplicated and likely need plugin writers to be involved.

Solution

Export

Make an export API that takes a 'repository_version' and it will generate a task that will create the export archive for the user. All Content types in the repository version, their Artifacts, and ContentArtifact objects are exported with django-import-export. This will make the file available for downloading via the pulp-content app somehow as a single archive.

Additionally the Export takes a 'prev_export_repository_version' option which will be used as the diff for all content between 'prev_export_repository_version' and 'repository_version'.

Import

Using the upload API the tarball can be uploaded in parallel efficiently. An import API that takes a 'repository' and it will generate a task that will import the archive for the user.

Technical Plan

To use django-import-export, plugin writers would provide a customized resource for their Content model and this would be put into a resources.py and imported by the Django plugin loader causing it to import for each plugin.

If the plugin writer doesn't provide one and the generic one will work for them, they do nothing, and pulpcore will create a generic one at Runtime

pulpcore would also provide Resources for Artifact and ContentArtifact which the import task would also handle.

Open Questions

How will users at import time specify if they want dry-run or not?


Related issues

Related to Pulp - Story #6134: [EPIC] Pulp import/exportCLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Has duplicate Ansible Plugin - Story #5088: support ansible collection content exportCLOSED - DUPLICATE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

History

#1 Updated by bmbouter over 1 year ago

  • Description updated (diff)

#2 Updated by bmbouter over 1 year ago

  • Description updated (diff)

#3 Updated by daviddavis over 1 year ago

  • Groomed changed from No to Yes
  • Sprint Candidate changed from No to Yes

#4 Updated by rchan over 1 year ago

  • Sprint set to Sprint 56

#5 Updated by bmbouter over 1 year ago

  • Has duplicate Story #5088: support ansible collection content export added

#6 Updated by bmbouter over 1 year ago

  • Tags Katello-P2 added

This duplicates a Katello P2 ( https://pulp.plan.io/issues/5088. ), so this should receive the Katello P2 label.

#7 Updated by rchan about 1 year ago

  • Sprint changed from Sprint 56 to Sprint 57

#8 Updated by rchan about 1 year ago

  • Sprint changed from Sprint 57 to Sprint 58

#9 Updated by rchan about 1 year ago

  • Sprint changed from Sprint 58 to Sprint 59

#10 Updated by daviddavis about 1 year ago

Removing from the sprint since this is not a 3.0 blocker.

#11 Updated by daviddavis about 1 year ago

  • Sprint deleted (Sprint 59)

#12 Updated by daviddavis about 1 year ago

  • Tags Pulp 2 to 3 Migrate added
  • Tags deleted (Katello-P2)

#13 Updated by rchan 9 months ago

  • Sprint Candidate deleted (Yes)

#14 Updated by jsherril@redhat.com 9 months ago

  • Sprint Candidate set to No
  • Tags Katello-P2 added

#15 Updated by daviddavis 8 months ago

  • Related to Story #6134: [EPIC] Pulp import/export added

#16 Updated by daviddavis 8 months ago

Superseded by #6134

#17 Updated by bmbouter 8 months ago

  • Status changed from NEW to CLOSED - DUPLICATE

closing as a duplicate of https://pulp.plan.io/issues/6134

#18 Updated by ggainey 6 months ago

  • Tags Katello added
  • Tags deleted (Katello-P2)

#19 Updated by bmbouter 5 months ago

  • Tags deleted (Pulp 2 to 3 Migrate)

We're removing the 'Pulp 2 to 3 Migrate' tag to allow membership of the Migration Plugin project in Redmine to be the controlling area https://pulp.plan.io/projects/migration

Please register to edit this issue

Also available in: Atom PDF