Story #5027
closed
As a user, I can 'docker push' to a Pulp repository
100%
Description
Motivation¶
There are currently two upload workflows for Pulp 3 users. Most plugins are encouraged to provide support for the latter.
Basic Upload¶
1. Create a repository named 'foo'
2. Upload an artifact
3. Create content unit
4. Create a new repository version of 'foo' from the content unit
Smart Upload¶
1. Create a repository name 'foo'
2. Upload content unit and create a repository version
The Smart Upload for the Docker plugin needs to support creating repository versions using a docker client.
Smart Upload for Docker¶
1. Create a repository named 'foo'
2. docker tag docker tag 0e5574283393 <pulp-hostname>:24817/foo
3. docker push <pulp-hostname>:24817/foo
Step 3 above creates a new repository version of 'foo' that contains a manifest '0e5574283393', a tag 'latest' pointing to the manifest, and all other manifests and blobs associated with manifest '0e5574283393'. The latest repository version of 'foo' can be consumed using docker client
4. docker pull <pulp-hostname>:24817/foo
Required REST APIs¶
/v2/ - GET¶
/v2/<repository-name>/blobs/<digest> - HEAD, GET¶
/v2/<repository-name>/blobs/<digest>/uploads - POST¶
/v2/<repository-name>/blobs/<digest>/uploads/<uuid> - HEAD, GET, PUT, PATCH¶
/v2/<repository-name>/manifests/<digest or tag> - HEAD, GET, PUT¶
etherpad for reference for the endpoints https://etherpad.net/p/docker_push
Related issues
Updated by dkliban@redhat.com over 5 years ago
- Tracker changed from Issue to Story
- % Done set to 0
The POC of the above APIs are here[0]. However, this only creates the content in Pulp. No new repository versions are being created. These changes also require creating a Docker Distribution. However, this story intentionally omits having the user create a docker distribution to simplify the workflow.
[0] https://github.com/pulp/pulp_docker/compare/master...dkliban:docker-push?expand=1
Updated by dkliban@redhat.com over 5 years ago
I updated the branch from the previous comment. However, it requires a change in pulpcore[0]. This change is currently only present on my fork of pulpcore[1].
The latest code creates a repository version on every push.
[0] https://pulp.plan.io/issues/5028
[1] https://github.com/dkliban/pulpcore/tree/5028-syncronous-repo-version-create
Updated by ipanova@redhat.com over 5 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to dkliban@redhat.com
- Sprint set to Sprint 55
Updated by ipanova@redhat.com over 5 years ago
- Project changed from Pulp to Docker Support
Updated by dkliban@redhat.com over 5 years ago
- Sprint changed from Sprint 55 to Sprint 56
Updated by dkliban@redhat.com over 5 years ago
- Status changed from ASSIGNED to NEW
The latest plan is to try and see if creating a temporary repository version for the duration of a docker push is a viable option. Then at the end of the push, when a tag is being created the repository version is created with a task.
Updated by dkliban@redhat.com over 5 years ago
During the last discussion about this feature we discussed a race condition. I'd like to discuss a possible solution.
Problem: docker push can result in corrupted repository if a blob, manifest, or manifest list is removed from the repository during the docker push workflow.
Solution: Perform validation of the content in the repository when the user makes the 'create tag' request. Return 400 error if any of the content is missing for that tag. This should check that the manifest list, manifest, and blobs are present. If a user receives a 400 error, she should try to push again. This time any content that went missing in the middle of the previous request will be re-uploaded.
This implies that we will create a new repository version with each blob/manifest/manifestlist/tag added to the repository. We would need to implement https://pulp.plan.io/issues/5028 as its description is currently written.
We could optionally squash repository versions when a tag is created. This would result in a single version for each tag added. This version would include all the manifests and blobks associated with the tag.
Updated by ipanova@redhat.com over 5 years ago
- Related to Story #4657: As a user i can use skopeo pulp transport to upload images to Pulp container registry added
Updated by dkliban@redhat.com over 5 years ago
- Assignee deleted (
dkliban@redhat.com)
Updated by ipanova@redhat.com about 5 years ago
Updated by ipanova@redhat.com about 5 years ago
- Project changed from Docker Support to Container Support
Updated by pulpbot over 4 years ago
- Status changed from NEW to POST
Added by dkliban@redhat.com over 4 years ago
Added by dkliban@redhat.com over 4 years ago
Revision 7f26e96a | View on GitHub
Adds REST APIs needed to perform docker/podman push
The user needs to create a Container Repository with a name that they will use to perform a push with. e.g. foo/bar
http :/pulp/api/v3/repositories/container/container/ name='foo/bar'
podman tag dhak235a localhost:24817/foo/bar podman push localhost:24817/foo/bar
Then the content can be pulled backed from Pulp by creating a distribution with the a base path that matches the name of the repository and has the repository associated with it. e.g. base_path='foo/bar' repository=
http :/pulp/api/v3/distributions/container/container/ name=foo base_path='foo/bar'
repository=/pulp/api/v3/repositories/container/container/1b5f0ac4-a911-42b8-bd36-4d497e6e9292/
podman pull localhost:24816/foo/bar
Note the difference in the port numbers of the two APIs.
Updated by pulpbot over 4 years ago
Updated by pulpbot over 4 years ago
Updated by ipanova@redhat.com about 4 years ago
- Status changed from POST to CLOSED - CURRENTRELEASE
.
Adds REST APIs needed to perform docker/podman push
The user needs to create a Container Repository with a name that they will use to perform a push with. e.g. foo/bar
http :/pulp/api/v3/repositories/container/container/ name='foo/bar'
podman tag dhak235a localhost:24817/foo/bar podman push localhost:24817/foo/bar
Then the content can be pulled backed from Pulp by creating a distribution with the a base path that matches the name of the repository and has the repository associated with it. e.g. base_path='foo/bar' repository=
http :/pulp/api/v3/distributions/container/container/ name=foo base_path='foo/bar'
repository=/pulp/api/v3/repositories/container/container/1b5f0ac4-a911-42b8-bd36-4d497e6e9292/
podman pull localhost:24816/foo/bar
Note the difference in the port numbers of the two APIs.
re: #5027 https://pulp.plan.io/issues/5027