Project

Profile

Help

Story #4716

As a user, I have simple content copy between repositories

Added by dalley over 1 year ago. Updated 12 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Sprint:
Sprint 53
Quarter:

Description

Simple content copy means that content units can be copied from one repository into another repository without considering content-type-specific invariants, such as making sure dependencies are satisfied.

The most straightforwards current design is to model this after one-shot upload. A new URL endpoint will be defined at /api/v3/rpm/copy/. POSTing to that endpoint with valid parameters will yield an asynchronous task, which will result in a new repository version in the target repository.

The user workflow should look like this:

http --form POST http://localhost:24817/pulp/api/v3/rpm/copy/ source_repository=${S_REPO_HREF} destination_repository=${D_REPO_HREF}

Both the repository parameters are required. When called, all content will be copied.

The additions to Django boilerplate will look something like this (see also: copy.py for an example of the function that does the work, and OneShotUploadSerializer in serializers.py)

urls.py

urlpatterns = [
    url(r'rpm/upload/$', OneShotUploadView.as_view()),
    url(r'rpm/copy/$', CopyView.as_view())
]

viewsets.py

class CopyView(views.APIView):
    def post(self, request):

        serializer = CopySerializer(data=request.data, context={'request': request})
        serializer.is_valid(raise_exception=True)

        # .....
        async_result = enqueue_with_reservation(
            copy_content, [source_repo, dest_repo],  # lock both repositories
            args=[source_repo, dest_repo],
            kwargs={}
        )
        return OperationPostponedResponse(async_result, request)

Related issues

Related to RPM Support - Test #4721: Test simple content copyNEW<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Related to RPM Support - Test #5276: Test - As a user, I have simple content copy between repositoriesNEW<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 0298593b View on GitHub
Added by dalley over 1 year ago

Add content copy feature

Also moved upload under app/tasks, because it's a task

closes #4716 https://pulp.plan.io/issues/4716

History

#1 Updated by dalley over 1 year ago

  • Description updated (diff)

#2 Updated by dalley over 1 year ago

  • Description updated (diff)

Questions to be answered at some point before merging:

  1. The exact names of the parameters, e.g. destination_repository vs target_repository, *repository vs. *repo, "content_set"
  2. Whether those is sufficient for the most basic implementation
  3. Whether doing a full copy when content_set is unspecified is appropriate

Questions for longer-term:

  1. Is this one endpoint sufficient for both errata and rpms, or will we need separate ones?
  2. Related to ^^, how do we implement filtering (e.g. copy "just" srpms, just rpms, just modules)
  3. Can this one endpoint be extended to cover the advanced copy cases, or should we make a separate one for those also?

#3 Updated by dalley over 1 year ago

  • Description updated (diff)

#4 Updated by bmbouter over 1 year ago

This story is well written. My only question has to do with content_set. If I'm a user and I'm specifying the list of content from the client side how is that functionally different than this API endpoint? https://docs.pulpproject.org/en/3.0/nightly/restapi.html#operation/repositories_versions_create

My suggestion is to remove the additional content_set option from this story's scope and let this copier have richer copy options added over time.

#5 Updated by dalley over 1 year ago

  • Groomed changed from No to Yes

Marking groomed for sprint planning, will adjust after

#6 Updated by daviddavis over 1 year ago

  • Sprint set to Sprint 52

#7 Updated by bmbouter over 1 year ago

  • Description updated (diff)

#8 Updated by dalley over 1 year ago

  • Description updated (diff)

#9 Updated by dalley over 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley

#10 Updated by dalley over 1 year ago

My assumption writing this issue is that we would just use the latest repository version from the source repo when copying.

Should we be taking a repository version as a source, instead of a repository? Or allow both? And do we need to allow changing the base_version to use on the destination repository, also?

#11 Updated by dalley over 1 year ago

  • Status changed from ASSIGNED to POST

#12 Updated by dalley over 1 year ago

  • Description updated (diff)

#13 Updated by dalley over 1 year ago

  • Related to Test #4721: Test simple content copy added

#14 Updated by ttereshc over 1 year ago

I agree that functionally it's the same as creating a new repo version when all the content is copied.
I see on the PR that the suggestion is to filter rpm specific content. +1 it will help for the cases with multiple remotes of different types. Though it's quite a rare case at the moment, imo.

Does it make sense to provide filtering by type right away? To have at least some additional functionality to repo version creation.

As for repo vs repo_version as a source:
I guess the latest version of a repo will satisfy most of use cases and it's convenient (for automation as well) to specify just a repo, at the same time it doesn't make sense to limit users to use the latest version only.
IIRC, we already have endpoints which accept either repo or repo_version, does it make sense to have the same here?
On the other hand, there are workarounds if we provide the option of the latest version only (use endpoint for repo verison creation, manually add/remove content units)

For the destination repo, I think it's enough to use the latest version.

#15 Updated by ipanova@redhat.com over 1 year ago

For source repo copy: I suggest allowing repository version or a repository ( like with publications)
For destination repo copy : I suggest accepting only latest version of the repo.
+1 to provide filtering by type right away

#16 Updated by bmbouter over 1 year ago

  • Tags deleted (Pulp 3)

#17 Updated by rchan over 1 year ago

  • Sprint changed from Sprint 52 to Sprint 53

#18 Updated by dalley over 1 year ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#19 Updated by kersom over 1 year ago

  • Related to Test #5276: Test - As a user, I have simple content copy between repositories added

#20 Updated by ttereshc 12 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF