Project

Profile

Help

Story #2810

closed

As a user, I can upload a schema 2 manifest with blobs

Added by mhrivnak almost 7 years ago. Updated over 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.15.0
Target Release - Docker:
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

Since docker's v2 registry API and associated data formats were released, it has not been possible to "docker save" a v2 image and retain the v2 data structures. Lacking a standard on-disk representation for a v2 image, it has not been possible for Pulp to receive an uploaded image.

This particularly impacts an image build workflow. An image must be built with docker engine, then pushed to a transient registry, and then sync'd by Pulp.

Skopeo [0] is a tool that can extract an image from docker engine directly and write it to disk. The on-disk representation is very simple (a manifest.json file, plus related blob files in a flat directory) and can be used as a de-facto standard.

A user will be able to:

  • "docker build" an image
  • "skopeo copy" that image to a directory
  • tar and upload that directory to Pulp

Pulp's docker plugin will need the importer to:

  • untar the upload file
  • recognize the file as containing a v2 schema 2 image
  • import the manifest and blobs
  • optionally receive a tag name to apply to the imported manifest

Skopeo does not yet support writing out schema 1 manifests to disk, so that is out of scope for this story.

This story does not require any changes to publishing or to crane.

After this story is successfully implemented, it will be possible to add Pulp as an destination type in skopeo. In that case, a user could do something along the lines of:

skopeo copy docker://myimage:2.0.0 pulp://example.com/myimage

The skopeo work is not part of this story, but it is useful for illustrating the value of this feature.

[0] https://github.com/projectatomic/skopeo

Actions #1

Updated by ipanova@redhat.com almost 7 years ago

w/r to the tags i think it worth mentioning in the story details, same for docs, that 'skopeo copy' looses all tags related to the copied repo, that's why user would need to optionally provide the tags during the upload for desired image manifests. Or he would have the chance to tag the image manifest later, once it was uploaded into Pulp.

Actions #2

Updated by ipanova@redhat.com almost 7 years ago

when you say 'Skopeo does not yet support writing out schema 1 manifests to disk' do you mean v1 schema1? I think skopeo does support v2 schema1 manifests, but just signed ones.

Actions #3

Updated by mansari over 6 years ago

  • Status changed from NEW to POST

Added by Ammar over 6 years ago

Revision cded2714 | View on GitHub

As a user, I can upload docker v2 schema 2 images saved via skopeo copy

closes #2810 https://pulp.plan.io/issues/2810

Added by Ammar over 6 years ago

Revision cded2714 | View on GitHub

As a user, I can upload docker v2 schema 2 images saved via skopeo copy

closes #2810 https://pulp.plan.io/issues/2810

Added by Ammar over 6 years ago

Revision cded2714 | View on GitHub

As a user, I can upload docker v2 schema 2 images saved via skopeo copy

closes #2810 https://pulp.plan.io/issues/2810

Added by Ammar over 6 years ago

Revision cded2714 | View on GitHub

As a user, I can upload docker v2 schema 2 images saved via skopeo copy

closes #2810 https://pulp.plan.io/issues/2810

Actions #4

Updated by Anonymous over 6 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #7

Updated by pcreech over 6 years ago

  • Platform Release set to 2.15.0
Actions #8

Updated by pcreech over 6 years ago

  • Status changed from MODIFIED to 5
Actions #9

Updated by Ichimonji10 over 6 years ago

@preethi I don't think this issue should be automated at this time. Doing so requires adding a dependency on skopeo, and we have enough other things to do without adding another dependency.

Actions #10

Updated by pcreech over 6 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #13

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added
Actions #14

Updated by Anonymous almost 4 years ago

Configs are essentially stored by the registry as normal blobs. They get referenced diversely in the show, however despite everything they're transferred by their summary and stored normally. Copper pipe Dubai

Also available in: Atom PDF