Project

Profile

Help

Story #5027

As a user, I can 'docker push' to a Pulp repository

Added by dkliban@redhat.com 5 months ago. Updated 8 days ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Platform Release:
Blocks Release:
Target Release - Docker:
Backwards Incompatible:
No
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

Motivation

There are currently two upload workflows for Pulp 3 users. Most plugins are encouraged to provide support for the latter.

Basic Upload

1. Create a repository named 'foo'
2. Upload an artifact
3. Create content unit
4. Create a new repository version of 'foo' from the content unit

Smart Upload

1. Create a repository name 'foo'
2. Upload content unit and create a repository version

The Smart Upload for the Docker plugin needs to support creating repository versions using a docker client.

Smart Upload for Docker

1. Create a repository named 'foo'
2. docker tag docker tag 0e5574283393 <pulp-hostname>:24817/foo
3. docker push <pulp-hostname>:24817/foo

Step 3 above creates a new repository version of 'foo' that contains a manifest '0e5574283393', a tag 'latest' pointing to the manifest, and all other manifests and blobs associated with manifest '0e5574283393'. The latest repository version of 'foo' can be consumed using docker client

4. docker pull <pulp-hostname>:24817/foo

Required REST APIs

/v2/ - GET

/v2/<repository-name>/blobs/<digest> - HEAD, GET

/v2/<repository-name>/blobs/<digest>/uploads - POST

/v2/<repository-name>/blobs/<digest>/uploads/<uuid> - HEAD, GET, PUT, PATCH

/v2/<repository-name>/manifests/<digest or tag> - HEAD, GET, PUT

etherpad for reference for the endpoints https://etherpad.net/p/docker_push


Related issues

Related to Docker Support - Story #4657: As a user i can use skopeo pulp transport to upload images to Pulp container registry NEW Actions

History

#1 Updated by dkliban@redhat.com 5 months ago

  • Tracker changed from Issue to Story
  • % Done set to 0

The POC of the above APIs are here0. However, this only creates the content in Pulp. No new repository versions are being created. These changes also require creating a Docker Distribution. However, this story intentionally omits having the user create a docker distribution to simplify the workflow.

[0] https://github.com/pulp/pulp_docker/compare/master...dkliban:docker-push?expand=1

#2 Updated by dkliban@redhat.com 5 months ago

I updated the branch from the previous comment. However, it requires a change in pulpcore0. This change is currently only present on my fork of pulpcore1.

The latest code creates a repository version on every push.

[0] https://pulp.plan.io/issues/5028
[1] https://github.com/dkliban/pulpcore/tree/5028-syncronous-repo-version-create

#3 Updated by ipanova@redhat.com 5 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dkliban@redhat.com
  • Sprint set to Sprint 55

#4 Updated by ipanova@redhat.com 5 months ago

  • Description updated (diff)

#5 Updated by ipanova@redhat.com 5 months ago

  • Project changed from Pulp to Docker Support

#6 Updated by dkliban@redhat.com 4 months ago

  • Sprint changed from Sprint 55 to Sprint 56

#7 Updated by dkliban@redhat.com 4 months ago

  • Status changed from ASSIGNED to NEW

The latest plan is to try and see if creating a temporary repository version for the duration of a docker push is a viable option. Then at the end of the push, when a tag is being created the repository version is created with a task.

#8 Updated by rchan 4 months ago

  • Sprint changed from Sprint 56 to Sprint 57

#9 Updated by dkliban@redhat.com 4 months ago

During the last discussion about this feature we discussed a race condition. I'd like to discuss a possible solution.

Problem: docker push can result in corrupted repository if a blob, manifest, or manifest list is removed from the repository during the docker push workflow.

Solution: Perform validation of the content in the repository when the user makes the 'create tag' request. Return 400 error if any of the content is missing for that tag. This should check that the manifest list, manifest, and blobs are present. If a user receives a 400 error, she should try to push again. This time any content that went missing in the middle of the previous request will be re-uploaded.

This implies that we will create a new repository version with each blob/manifest/manifestlist/tag added to the repository. We would need to implement https://pulp.plan.io/issues/5028 as its description is currently written.

We could optionally squash repository versions when a tag is created. This would result in a single version for each tag added. This version would include all the manifests and blobks associated with the tag.

#10 Updated by ipanova@redhat.com 3 months ago

  • Related to Story #4657: As a user i can use skopeo pulp transport to upload images to Pulp container registry added

#11 Updated by dkliban@redhat.com 3 months ago

  • Assignee deleted (dkliban@redhat.com)

#12 Updated by rchan 3 months ago

  • Sprint changed from Sprint 57 to Sprint 58

#13 Updated by ipanova@redhat.com 3 months ago

  • Sprint deleted (Sprint 58)

Please register to edit this issue

Also available in: Atom PDF