Project

Profile

Help

Story #1051

As a user, I can publish v2 repositories

Added by mhrivnak over 5 years ago. Updated over 1 year ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Target Release - Docker:
2.0.0
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Quarter:

Description

Deliverables

  • web distributor can publish manifests and related v2 content
  • export distributor can publish manifests and related v2 content
drop_database.js (18 Bytes) drop_database.js rbarlow, 08/17/2015 09:14 PM
setup_v2.sh (1.91 KB) setup_v2.sh rbarlow, 08/17/2015 09:14 PM

Related issues

Related to Docker Support - Story #1444: As a Docker user, I receive the Docker-Content-Digest header from the Manifest GET callCLOSED - WONTFIX

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Blocked by Crane - Story #1049: Add v2 concepts to data file schemaCLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Blocks Docker Support - Task #1048: Tracker for Docker v2 API and Manifest workCLOSED - CURRENTRELEASE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

Revision 4d3a7dd9 View on GitHub
Added by rbarlow about 5 years ago

Add support for publishing Docker v2 content.

This commit adds support for publishing Docker v2 content.

https://pulp.plan.io/issues/1051

closes #1051

Revision 4d3a7dd9 View on GitHub
Added by rbarlow about 5 years ago

Add support for publishing Docker v2 content.

This commit adds support for publishing Docker v2 content.

https://pulp.plan.io/issues/1051

closes #1051

Revision 4d3a7dd9 View on GitHub
Added by rbarlow about 5 years ago

Add support for publishing Docker v2 content.

This commit adds support for publishing Docker v2 content.

https://pulp.plan.io/issues/1051

closes #1051

Revision 4d3a7dd9 View on GitHub
Added by rbarlow about 5 years ago

Add support for publishing Docker v2 content.

This commit adds support for publishing Docker v2 content.

https://pulp.plan.io/issues/1051

closes #1051

History

#1 Updated by mhrivnak over 5 years ago

  • Blocked by Story #1049: Add v2 concepts to data file schema added

#2 Updated by mhrivnak over 5 years ago

  • Blocks Task #1048: Tracker for Docker v2 API and Manifest work added

#3 Updated by mhrivnak about 5 years ago

  • Groomed changed from No to Yes

#4 Updated by mhrivnak about 5 years ago

  • Priority changed from Normal to High

#5 Updated by rbarlow about 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to rbarlow

v1 repositories have repository tags and v2 repositories have manifests. Nothing identifies whether a given repository is v1 or v2, so the plan is to loop over all the tags in a repo and publish them as v1 content, and then to loop over the manifests in a repo and publish them as v2 content. This does mean that a repository can have v1 and v2 content at the same time, and that the content won't necessarily be the same. However, it advantageous because it keeps the design simpler.

It was also noted that v2 content can be published by Pulp in a format that is acceptable to the docker client so we will do that as part of this effort as well.

#6 Updated by rbarlow about 5 years ago

I've attached a couple of scripts that help you quickly go from a Vagrant up in a master checkout of all Pulp repositories to a running Docker v2 against Pulp 2.6. To use them, I recommend putting them in your pulp_docker folder on the host and then:

0) vagrant up
1) Make some coffee
2) vagrant ssh
3) workon pulp_docker
4) time ./setup_v2.sh
5) Drink said coffee (step 4 takes about 4 minutes on my spinny disk computerbox)
6) Profit
6a) You can now run pulp-admin docker repo sync run --repo-id v1_busybox to make sure v1 publish still works
6b) You can now run pulp-admin docker repo sync run --repo-id v2_registry to test your new v2 publish code

#7 Updated by rbarlow about 5 years ago

  • Status changed from ASSIGNED to POST

#8 Updated by rbarlow about 5 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#9 Updated by rbarlow about 5 years ago

  • Status changed from MODIFIED to ASSIGNED
  • % Done changed from 100 to 0

This story is not fully complete, as there is one step left to solve. We need some way to inject the Docker-Content-Digest header into the Manifest GET handler, as described at https://github.com/docker/distribution/blob/release/2.0/docs/spec/api.md#get-manifest . Without that header, Pulp cannot sync from itself and I suspect that the Docker client will not work with Pulp either.

I was planning to use htaccess files to add the Docker-Content-Digest headers. I'm not completely sure this will work, but I believe it has a good chance of solving the problem. If this works, we can have the Distributor create an htaccess file when publishing the manifests.

#10 Updated by rbarlow about 5 years ago

This Docker bug is why the current publish code that we have is not sufficient for the docker pull to work against Pulp as is: https://github.com/docker/docker/issues/15173

#11 Updated by jortel@redhat.com about 5 years ago

rbarlow wrote:

This story is not fully complete, as there is one step left to solve. We need some way to inject the Docker-Content-Digest header into the Manifest GET handler, as described at https://github.com/docker/distribution/blob/release/2.0/docs/spec/api.md#get-manifest . Without that header, Pulp cannot sync from itself and I suspect that the Docker client will not work with Pulp either.

I was planning to use htaccess files to add the Docker-Content-Digest headers. I'm not completely sure this will work, but I believe it has a good chance of solving the problem. If this works, we can have the Distributor create an htaccess file when publishing the manifests.

A possible solution would be to use the asis directive and publish the manifest exactly as it will be served including the header. Not sold but wanted to document this as a possibility.

[1] http://httpd.apache.org/docs/2.2/mod/mod_asis.html

#12 Updated by rbarlow almost 5 years ago

  • Target Release - Docker set to 2.0.0

#13 Updated by rbarlow almost 5 years ago

  • Related to Story #1444: As a Docker user, I receive the Docker-Content-Digest header from the Manifest GET call added

#14 Updated by rbarlow almost 5 years ago

  • Status changed from ASSIGNED to MODIFIED
  • % Done changed from 0 to 100

I filed a story about figuring out how to serve the Manifest's digest header, so we can call this work "done".

#15 Updated by rbarlow over 4 years ago

  • Status changed from MODIFIED to 5

#16 Updated by rbarlow over 4 years ago

I'm gonna pulp-smash this mammajamma.

#17 Updated by rbarlow over 4 years ago

This PR adds tests to verify this bug: https://github.com/PulpQE/pulp-smash/pull/139

#18 Updated by rbarlow over 4 years ago

  • Status changed from 5 to 6

Verified by pulp-smash.

#19 Updated by dkliban@redhat.com over 4 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE

#21 Updated by bmbouter over 1 year ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF