Project

Profile

Help

Story #4244

closed

Convert manifest to schema1 when older clients are pulling by tag

Added by amacdona@redhat.com about 6 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
(Total: 0:00 h)
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 3 docker blocker
Sprint:
Sprint 61
Quarter:

Description

Problem:

When pulling newer format manifest by tag old clients are not supported.

Solution:

Rewrite available manifest to schema1 to support old clients.

Workflow:

Fetch the available manifest from the storage backend, if it exists. Parse incoming accept headers from client (client indicates support for certain manifest formats) and if necessary rewrite manifest when it is being fetched by tag. If it is being fetched by digest, conversion is not possible.

  • If available manifest is a manifest list - within the manifest list find the image manifest corresponding to the default platform and arch (amd64 platform and linux OS). If no suitable manifest is found in the manifest list, return a 404 error.
    If necessary( based on incoming accept headers), convert the image manifest to schema1. Invoke Schema1ManifestBuilder to create skeleton of schema1 format. Populate the builder with the data incoming from parsed image manifest schema2 json.
    Sign schema1 manifest with the provided signing key (in config). If no signing key is provided generate an ephemeral rsa key to be used for signing converted manifests.
  • If available manifest is a manifest schema2 , if necessary( based on incoming accept headers), convert the image manifest to schema1.
  • If available manifest is a manifest schema1, return as is.

Signed Manifest Field Description

Signed manifests include an image manifest and a list of signatures. A signature consists of the following fields:

header JOSE
A JSON Web Signature
signature string
A signature for the image manifest, signed by a private key
protected string
The signed protected header

Optional conversion

The conversion will be optional. There will be a boolean flag called for example 'schema_conversion` added as a parameter to the docker distribution [0] where its default value would be set to False.
Conversion would happen only if enabled.

[0] https://github.com/pulp/pulp_docker/blob/master/pulp_docker/app/models.py#L297
https://docs.docker.com/registry/spec/manifest-v2-2/#backward-

Opened questions:

Do we convert manifest schema2 if it had foreign layers?


Sub-issues 1 (0 open1 closed)

Story #5635: Implement Schema 2 to Schema 1 converterCLOSED - CURRENTRELEASEmihai.ibanescu@gmail.com

Actions

Related issues

Is duplicate of Container Support - Issue #5146: Docker sync missing schema version 1 manifestsCLOSED - DUPLICATEActions
Actions #1

Updated by amacdona@redhat.com about 6 years ago

  • Description updated (diff)
Actions #2

Updated by amacdona@redhat.com about 6 years ago

I think this affects the Tag view and the Manifest/ManifestList views.

Actions #3

Updated by CodeHeeler about 6 years ago

  • Triaged changed from No to Yes
Actions #4

Updated by ipanova@redhat.com over 5 years ago

  • Tags Pulp 3 added
Actions #5

Updated by ipanova@redhat.com over 5 years ago

  • Tracker changed from Issue to Story
  • Subject changed from Support old (v2s2) clients that do not support manifest lists to Support old clients that do not support v2s2
  • Description updated (diff)
  • % Done set to 0
Actions #6

Updated by ipanova@redhat.com over 5 years ago

  • Description updated (diff)
Actions #7

Updated by ipanova@redhat.com over 5 years ago

  • Description updated (diff)
Actions #8

Updated by ipanova@redhat.com over 5 years ago

  • Subject changed from Support old clients that do not support v2s2 to Support old clients that do not support v2s2 when pulling by tag
  • Description updated (diff)
Actions #9

Updated by bmbouter over 5 years ago

  • Tags deleted (Pulp 3)
Actions #10

Updated by ipanova@redhat.com over 5 years ago

  • Description updated (diff)
Actions #11

Updated by ipanova@redhat.com over 5 years ago

Shall we make this conversion optional? we could introduce a flag to the docker distribution where its default value would be set to False.
Conversion would happen only if enabled.

Actions #12

Updated by ipanova@redhat.com over 5 years ago

  • Description updated (diff)
Actions #13

Updated by ipanova@redhat.com over 5 years ago

  • Description updated (diff)
Actions #14

Updated by ipanova@redhat.com over 5 years ago

  • Description updated (diff)
Actions #15

Updated by ipanova@redhat.com over 5 years ago

  • Is duplicate of Issue #5146: Docker sync missing schema version 1 manifests added
Actions #16

Updated by ipanova@redhat.com over 5 years ago

  • Description updated (diff)
Actions #17

Updated by ipanova@redhat.com over 5 years ago

  • Subject changed from Support old clients that do not support v2s2 when pulling by tag to Convert manifest to schema1 when older clients are pulling by tag
Actions #18

Updated by ipanova@redhat.com over 5 years ago

  • Description updated (diff)
Actions #19

Updated by ipanova@redhat.com over 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ipanova@redhat.com
Actions #20

Updated by ipanova@redhat.com over 5 years ago

  • Sprint set to Sprint 56
Actions #21

Updated by rchan over 5 years ago

  • Sprint changed from Sprint 56 to Sprint 57
Actions #22

Updated by ipanova@redhat.com over 5 years ago

  • Status changed from ASSIGNED to NEW
Actions #23

Updated by rchan over 5 years ago

  • Sprint changed from Sprint 57 to Sprint 58
Actions #24

Updated by ipanova@redhat.com over 5 years ago

  • Sprint deleted (Sprint 58)
Actions #25

Updated by ipanova@redhat.com about 5 years ago

  • Tags Pulp 3 docker blocker added
Actions #26

Updated by ipanova@redhat.com about 5 years ago

  • Assignee deleted (ipanova@redhat.com)
Actions #27

Updated by ipanova@redhat.com about 5 years ago

  • Sprint set to Sprint 61
Actions #29

Updated by lmjachky about 5 years ago

  • Status changed from NEW to POST
  • Assignee set to lmjachky

Added by Lubos Mjachky about 5 years ago

Revision 42183b9c | View on GitHub

Execute implemented schema conversion from the registry

closes #4244 https://pulp.plan.io/issues/4244

Added by Lubos Mjachky about 5 years ago

Revision 42183b9c | View on GitHub

Execute implemented schema conversion from the registry

closes #4244 https://pulp.plan.io/issues/4244

Added by Lubos Mjachky about 5 years ago

Revision 42183b9c | View on GitHub

Execute implemented schema conversion from the registry

closes #4244 https://pulp.plan.io/issues/4244

Added by Lubos Mjachky about 5 years ago

Revision 42183b9c | View on GitHub

Execute implemented schema conversion from the registry

closes #4244 https://pulp.plan.io/issues/4244

Actions #30

Updated by Anonymous about 5 years ago

  • Status changed from POST to MODIFIED

Applied in changeset commit:pulp_docker|42183b9cea99a2bc244e7c4ca9363cbf5a4121c9.

Actions #31

Updated by ipanova@redhat.com about 5 years ago

  • Project changed from Docker Support to Container Support
Actions #32

Updated by ipanova@redhat.com about 5 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #33

Updated by ipanova@redhat.com almost 5 years ago

  • Sprint/Milestone set to 1.0.0

Also available in: Atom PDF