Project

Profile

Help

Maintenance: Planio will be observing a scheduled maintenance window this Tuesday, November 5, 2024 from 03:00 UTC until 06:30 UTC to perform urgent network maintenance in our primary data center. Your Planio account will be unavailable during this maintenance window.

Issue #6018

closed

Story #4762: [Epic] As a user, I can copy content

As a user, I have an API for copying content between repositories

Added by dalley almost 5 years ago. Updated over 4 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
Estimated time:
(Total: 0:00 h)
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 68
Quarter:

Description

API Design

The problems created by multi-repo-copy and repository versions in particular necessitates an entirely JSON-blob-based copy configuration.

  • We must support an arbitrary number of source and destination repositories (multi-repo-copy)
    • We must be able to simultaneously filter and copy from all of the source repos (not just the "primary" one like in Pulp 2) for performance reasons
    • We can't provide a global content list/criteria to search the source repos for for correctness reasons [0], each pair of source/destination must have its own content list/criteria to use
  • We must support setting "base_version" on each of the created repository versions in the destination repositories
    • Katello can't just use the latest versions, sometimes they have to pick different versions of the destination repositories to use as bases.

So for every pair of source and destination repos, of which there can be an artibtrary number, you also have to be able to (optionally) provide a list of content/criteria to use for the copy and (optionally) use a different "base_version". Trying to do all of those things at once with combinations of parameters would be not be a reasonable thing to do, but, it's not so difficult with JSON.

[0] If multiple repos contain the same content unit, Pulp can't decide which one to copy from. With multiple destination repositories, this is important because the source repository determines the destination repository

MVP Copy API Concept

POST /pulp/api/v3/rpm/copy/ 
    config:=[
        {"source_repo_version": "$SRC_REPO_VERS_HREF", "dest_repo": "$DEST_REPO_HREF", "content": [$HREF1, $HREF2]},
        {"source_repo_version": "$SRC_REPO_VERS_HREF", "dest_repo": "$DEST_REPO_HREF", "dest_base_version": "$DEST_BASE_VERSION", "content": []},
    ]
    dependency_solving=False

There are some obvious extensions that could be made for ergonomics and usability but they are not strictly necessary for the MVP, and they're pure extensions. Due to time constraints on integration, we should probably stick to the MVP and add the other niceties later.

Endpoint schema

"config" (required, list): Complex configuration object "dependency_solving" (optional, bool): Enable or disable dependency solving for the copy operation. Default TBD based on performance.

JSON config schema

The config is a list of dictionaries where the keys are roughly the parameters you would normally expect for a copy operation

"source_repo_version" (required, repository version href): If provided, uses this repository version as the source content set "dest_repo" (required, repository href): The repository to create a new version in (copy content into) "dest_base_version" (optional, int): The base version to use for the destination repository "content" (TBD, list): The list of content hrefs to be copied


Sub-issues 2 (0 open2 closed)

Story #6019: As a user, I can specify content to be copied by the content's pulp_hrefCLOSED - CURRENTRELEASEdaviddavis

Actions
Story #6268: As a user, I can copy from multiple source repo versions to destination repos at one timeCLOSED - CURRENTRELEASEdaviddavis

Actions
Actions #1

Updated by rchan almost 5 years ago

  • Sprint changed from Sprint 64 to Sprint 65
Actions #2

Updated by dalley over 4 years ago

  • Triaged changed from No to Yes
Actions #3

Updated by rchan over 4 years ago

  • Sprint changed from Sprint 65 to Sprint 66
Actions #4

Updated by rchan over 4 years ago

  • Sprint changed from Sprint 66 to Sprint 67
Actions #5

Updated by rchan over 4 years ago

  • Sprint changed from Sprint 67 to Sprint 68
Actions #6

Updated by dalley over 4 years ago

  • Status changed from ASSIGNED to POST
  • Assignee changed from dalley to daviddavis
Actions #7

Updated by dalley over 4 years ago

  • Description updated (diff)
Actions #8

Updated by daviddavis over 4 years ago

  • Status changed from POST to MODIFIED
Actions #9

Updated by daviddavis over 4 years ago

Actions #10

Updated by daviddavis over 4 years ago

Actions #11

Updated by daviddavis over 4 years ago

Actions #12

Updated by daviddavis over 4 years ago

Actions #13

Updated by daviddavis over 4 years ago

Actions #14

Updated by daviddavis over 4 years ago

Added by dalley over 4 years ago

Revision c3c3c491 | View on GitHub

Initial copy implementation

Criteria and depsolving are no-op -- copies everything

Also includes a basic smash test

re: #6018 https://pulp.plan.io/issues/6018

Added by daviddavis over 4 years ago

Revision 09187398 | View on GitHub

Allow copying of multiple sources to destinations in one call

fixes #6268 https://pulp.plan.io/issues/6268

fixes #6018 https://pulp.plan.io/issues/6018

Actions #15

Updated by daviddavis over 4 years ago

Actions #16

Updated by dalley over 4 years ago

  • Sprint/Milestone changed from Pulp 3.x RPM (Katello 3.16) to Pulp RPM 3.3.0
Actions #17

Updated by dalley over 4 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Also available in: Atom PDF