Project

Profile

Help

Story #1051

closed

As a user, I can publish v2 repositories

Added by mhrivnak over 9 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Target Release - Docker:
2.0.0
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Quarter:

Description

Deliverables

  • web distributor can publish manifests and related v2 content
  • export distributor can publish manifests and related v2 content

Files

drop_database.js (18 Bytes) drop_database.js rbarlow, 08/17/2015 09:14 PM
setup_v2.sh (1.91 KB) setup_v2.sh rbarlow, 08/17/2015 09:14 PM

Related issues

Related to Docker Support - Story #1444: As a Docker user, I receive the Docker-Content-Digest header from the Manifest GET callCLOSED - WONTFIX

Actions
Blocked by Crane - Story #1049: Add v2 concepts to data file schemaCLOSED - CURRENTRELEASErbarlow

Actions
Blocks Docker Support - Task #1048: Tracker for Docker v2 API and Manifest workCLOSED - CURRENTRELEASErbarlow

Actions
Actions #1

Updated by mhrivnak over 9 years ago

  • Blocked by Story #1049: Add v2 concepts to data file schema added
Actions #2

Updated by mhrivnak over 9 years ago

  • Blocks Task #1048: Tracker for Docker v2 API and Manifest work added
Actions #3

Updated by mhrivnak over 9 years ago

  • Groomed changed from No to Yes
Actions #4

Updated by mhrivnak over 9 years ago

  • Priority changed from Normal to High
Actions #5

Updated by rbarlow over 9 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to rbarlow

v1 repositories have repository tags and v2 repositories have manifests. Nothing identifies whether a given repository is v1 or v2, so the plan is to loop over all the tags in a repo and publish them as v1 content, and then to loop over the manifests in a repo and publish them as v2 content. This does mean that a repository can have v1 and v2 content at the same time, and that the content won't necessarily be the same. However, it advantageous because it keeps the design simpler.

It was also noted that v2 content can be published by Pulp in a format that is acceptable to the docker client so we will do that as part of this effort as well.

Actions #6

Updated by rbarlow over 9 years ago

I've attached a couple of scripts that help you quickly go from a Vagrant up in a master checkout of all Pulp repositories to a running Docker v2 against Pulp 2.6. To use them, I recommend putting them in your pulp_docker folder on the host and then:

0) vagrant up
1) Make some coffee
2) vagrant ssh
3) workon pulp_docker
4) time ./setup_v2.sh
5) Drink said coffee (step 4 takes about 4 minutes on my spinny disk computerbox)
6) Profit
6a) You can now run pulp-admin docker repo sync run --repo-id v1_busybox to make sure v1 publish still works
6b) You can now run pulp-admin docker repo sync run --repo-id v2_registry to test your new v2 publish code

Added by rbarlow over 9 years ago

Revision 4d3a7dd9 | View on GitHub

Add support for publishing Docker v2 content.

This commit adds support for publishing Docker v2 content.

https://pulp.plan.io/issues/1051

closes #1051

Added by rbarlow over 9 years ago

Revision 4d3a7dd9 | View on GitHub

Add support for publishing Docker v2 content.

This commit adds support for publishing Docker v2 content.

https://pulp.plan.io/issues/1051

closes #1051

Added by rbarlow over 9 years ago

Revision 4d3a7dd9 | View on GitHub

Add support for publishing Docker v2 content.

This commit adds support for publishing Docker v2 content.

https://pulp.plan.io/issues/1051

closes #1051

Added by rbarlow over 9 years ago

Revision 4d3a7dd9 | View on GitHub

Add support for publishing Docker v2 content.

This commit adds support for publishing Docker v2 content.

https://pulp.plan.io/issues/1051

closes #1051

Actions #7

Updated by rbarlow over 9 years ago

  • Status changed from ASSIGNED to POST
Actions #8

Updated by rbarlow over 9 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #9

Updated by rbarlow over 9 years ago

  • Status changed from MODIFIED to ASSIGNED
  • % Done changed from 100 to 0

This story is not fully complete, as there is one step left to solve. We need some way to inject the Docker-Content-Digest header into the Manifest GET handler, as described at https://github.com/docker/distribution/blob/release/2.0/docs/spec/api.md#get-manifest . Without that header, Pulp cannot sync from itself and I suspect that the Docker client will not work with Pulp either.

I was planning to use htaccess files to add the Docker-Content-Digest headers. I'm not completely sure this will work, but I believe it has a good chance of solving the problem. If this works, we can have the Distributor create an htaccess file when publishing the manifests.

Actions #10

Updated by rbarlow over 9 years ago

This Docker bug is why the current publish code that we have is not sufficient for the docker pull to work against Pulp as is: https://github.com/docker/docker/issues/15173

Actions #11

Updated by jortel@redhat.com over 9 years ago

rbarlow wrote:

This story is not fully complete, as there is one step left to solve. We need some way to inject the Docker-Content-Digest header into the Manifest GET handler, as described at https://github.com/docker/distribution/blob/release/2.0/docs/spec/api.md#get-manifest . Without that header, Pulp cannot sync from itself and I suspect that the Docker client will not work with Pulp either.

I was planning to use htaccess files to add the Docker-Content-Digest headers. I'm not completely sure this will work, but I believe it has a good chance of solving the problem. If this works, we can have the Distributor create an htaccess file when publishing the manifests.

A possible solution would be to use the asis directive and publish the manifest exactly as it will be served including the header. Not sold but wanted to document this as a possibility.

[1] http://httpd.apache.org/docs/2.2/mod/mod_asis.html

Actions #12

Updated by rbarlow about 9 years ago

  • Target Release - Docker set to 2.0.0
Actions #13

Updated by rbarlow about 9 years ago

  • Related to Story #1444: As a Docker user, I receive the Docker-Content-Digest header from the Manifest GET call added
Actions #14

Updated by rbarlow about 9 years ago

  • Status changed from ASSIGNED to MODIFIED
  • % Done changed from 0 to 100

I filed a story about figuring out how to serve the Manifest's digest header, so we can call this work "done".

Actions #15

Updated by rbarlow almost 9 years ago

  • Status changed from MODIFIED to 5
Actions #16

Updated by rbarlow almost 9 years ago

I'm gonna pulp-smash this mammajamma.

Actions #17

Updated by rbarlow almost 9 years ago

This PR adds tests to verify this bug: https://github.com/PulpQE/pulp-smash/pull/139

Actions #18

Updated by rbarlow almost 9 years ago

  • Status changed from 5 to 6

Verified by pulp-smash.

Actions #19

Updated by dkliban@redhat.com almost 9 years ago

  • Status changed from 6 to CLOSED - CURRENTRELEASE
Actions #21

Updated by bmbouter over 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF