Project

Profile

Help

Story #2810

As a user, I can upload a schema 2 manifest with blobs

Added by mhrivnak over 4 years ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.15.0
Target Release - Docker:
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Since docker's v2 registry API and associated data formats were released, it has not been possible to "docker save" a v2 image and retain the v2 data structures. Lacking a standard on-disk representation for a v2 image, it has not been possible for Pulp to receive an uploaded image.

This particularly impacts an image build workflow. An image must be built with docker engine, then pushed to a transient registry, and then sync'd by Pulp.

Skopeo [0] is a tool that can extract an image from docker engine directly and write it to disk. The on-disk representation is very simple (a manifest.json file, plus related blob files in a flat directory) and can be used as a de-facto standard.

A user will be able to:

  • "docker build" an image
  • "skopeo copy" that image to a directory
  • tar and upload that directory to Pulp

Pulp's docker plugin will need the importer to:

  • untar the upload file
  • recognize the file as containing a v2 schema 2 image
  • import the manifest and blobs
  • optionally receive a tag name to apply to the imported manifest

Skopeo does not yet support writing out schema 1 manifests to disk, so that is out of scope for this story.

This story does not require any changes to publishing or to crane.

After this story is successfully implemented, it will be possible to add Pulp as an destination type in skopeo. In that case, a user could do something along the lines of:

skopeo copy docker://myimage:2.0.0 pulp://example.com/myimage

The skopeo work is not part of this story, but it is useful for illustrating the value of this feature.

[0] https://github.com/projectatomic/skopeo

Associated revisions

Revision cded2714 View on GitHub
Added by Ammar over 4 years ago

As a user, I can upload docker v2 schema 2 images saved via skopeo copy

closes #2810 https://pulp.plan.io/issues/2810

Revision cded2714 View on GitHub
Added by Ammar over 4 years ago

As a user, I can upload docker v2 schema 2 images saved via skopeo copy

closes #2810 https://pulp.plan.io/issues/2810

Revision cded2714 View on GitHub
Added by Ammar over 4 years ago

As a user, I can upload docker v2 schema 2 images saved via skopeo copy

closes #2810 https://pulp.plan.io/issues/2810

Revision cded2714 View on GitHub
Added by Ammar over 4 years ago

As a user, I can upload docker v2 schema 2 images saved via skopeo copy

closes #2810 https://pulp.plan.io/issues/2810

History

#1 Updated by ipanova@redhat.com over 4 years ago

w/r to the tags i think it worth mentioning in the story details, same for docs, that 'skopeo copy' looses all tags related to the copied repo, that's why user would need to optionally provide the tags during the upload for desired image manifests. Or he would have the chance to tag the image manifest later, once it was uploaded into Pulp.

#2 Updated by ipanova@redhat.com over 4 years ago

when you say 'Skopeo does not yet support writing out schema 1 manifests to disk' do you mean v1 schema1? I think skopeo does support v2 schema1 manifests, but just signed ones.

#3 Updated by mansari over 4 years ago

  • Status changed from NEW to POST

#4 Updated by Anonymous over 4 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#7 Updated by pcreech about 4 years ago

  • Platform Release set to 2.15.0

#8 Updated by pcreech about 4 years ago

  • Status changed from MODIFIED to 5

#9 Updated by Ichimonji10 about 4 years ago

@preethi I don't think this issue should be automated at this time. Doing so requires adding a dependency on skopeo, and we have enough other things to do without adding another dependency.

#10 Updated by pcreech about 4 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

#13 Updated by bmbouter almost 3 years ago

  • Tags Pulp 2 added

#14 Updated by Anonymous over 1 year ago

Configs are essentially stored by the registry as normal blobs. They get referenced diversely in the show, however despite everything they're transferred by their summary and stored normally. Copper pipe Dubai

Also available in: Atom PDF