Docker Support

Can you elaborate on the openshift need for schema 2? After investigating schema 2, we determined in consultation with @vbatts that all docker-related tooling should still be compatible with schema 1, and there should not be a need to support both.

Discussed this with bleanhar@redhat.com today and I think this came from a misunderstanding.

However, thinking around the issue today I realised something else relating to it.

Once we update our container image builder to docker 1.10, we will be pushing images to a docker-distribution instance and these images will have V2 schema 2 manifests. Our build system notes the manifest digest reported by docker after the push, then tells Pulp to sync from docker-distribution.

However, the digest noted by the build system and stored in Koji for the build will be different to the one available in Crane: after 'docker push' the digest is for a schema 2 manifest, but after 'sync' the manifest digest Crane will know will be for a schema 1 manifest.

To work around this we will need to query the docker-distribution instance for the schema 1 manifest and note the Docker-Content-Digest header value.

Is support for V2 schema 2 on the roadmap for pulp_docker/crane? I have an idea about how it could work.

Can you please specify if you need just schema v2 support of regular image manifest or you also need support manifests list, so called 'fat manifest'?

Discussed it with Amanda: currently we need just schema 2 (as we're planning to migrate to Docker 1.10 soon).

Note that multiarch implementation is also planned (thus fat manifest is needed as well), but it first has to be discussed with Content Delivery team

To support schema 1 and 2, we would need to:

• change the manifest DB model to accomodate schema 2
• retrieve both schema 1 and 2 during sync and save both
• change the tag model to store both schema 1 and schema 2 manifest digests on the same instance, or add a schema field as part of the unit key
• possibly change the tag copy logic to make sure it's replacing tag models correctly now that there are two digests to worry about
• change the schema of the crane metadata file to have two manifest lists: one for schema 1, and one for schema 2
• change publish to create the new crane metadata file
• change crane to look for a optional header from the client and use that to decide whether to return a schema 1 or schema 2 manifest

@ipanova looked up that docker version 1.9 and older only do schema 1. We would consider just dropping schema 1 support, but that probably would not be acceptable at this time to only support docker 1.10+.

This issue is at least loosely related to this atomic registry bug:

https://github.com/openshift/origin/issues/10730

which was resolved here:

https://github.com/openshift/origin/pull/10805

and the fix appears to be available already upstream. Pulp has been tested syncing against that by @asmacdo

---N.B. 'docker repo tag' is broken. needs to be updated, --schema-version needs to be added as another required-field---

This requirement was doen on server side. From client perspective nothing changes

N.B docker rsync distributor needs to be modified as well

Docker rsync dist publish path was changed because of publish directory structure change in general

ipanova@redhat.com wrote:

N.B docker rsync distributor needs to be modified as well

Docker rsync dist publish path was changed because of publish directory structure change in general

We solved this by adding to apache config to set proper response headers based on directory match

This issue needs additional verification. See: https://github.com/PulpQE/pulp-smash/issues/555#issuecomment-297168241