Issue #6814
closeddocker-content-digest header is missing from the response on s3 files
Description
we have set content-type and content-disposition headers, but we also need docker-content-digest present among response headers
usecase: Users want to make a HEAD request for a tag to find out if that tag is pointing to a new manifest. Without this header, users must download the manifest in order to learn its digest.
Updated by ipanova@redhat.com over 4 years ago
- Triaged changed from No to Yes
- Sprint set to Sprint 73
Updated by mdellweg over 4 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to mdellweg
Updated by mdellweg over 4 years ago
A little bit of poking into the registry api at quay lets me to believe that is is sufficient to hand over the Docker-Content-Digest
header with the redirect response (assuming they have found a proper way to handle it):
$ http --header https://quay.io/v2/pulp/pulp-fixtures/blobs/sha256:55f96fcd2154dea0366210d95996aa7f17c3a1d5a8ff86da075c038b56fe17b5
HTTP/1.1 302 FOUND
Accept-Ranges: bytes
Cache-Control: max-age=31536000
Connection: close
Content-Length: 1225
Content-Type: text/html; charset=utf-8
Date: Tue, 26 May 2020 11:21:26 GMT
Docker-Content-Digest: sha256:55f96fcd2154dea0366210d95996aa7f17c3a1d5a8ff86da075c038b56fe17b5
Location: https://cdn.quay.io/sha256/55/55f96fcd2154dea0366210d95996aa7f17c3a1d5a8ff86da075c038b56fe17b5?Expires=1590492686&Signature=MttvuW56~Obzu-t5lz6w8Xx8BFYMeWX3NekRdNHm0aGS1dncw7lBo5uzUB1LKtXxWM3gsK31-StT56rp9N05cKCzvNpw1oYrPslLdiLBBErexw~DJwYrDENheza-2AfO045IN7PS20NejpTTece55srCkXBRjZvdMM4N5AKSkFfVORST4AfPjaWjQenB9wyHQgca9Ux3EmK2UsYAhb953rjBA5sdo6MZV-I87CYHwCggsOwioG33PuGZV5LhgXGpjOUeA1QQPBE1a3M-MsvQ5bvvsWMinYRA7nPsnhyihMfomrL2VtTTyG7foeLAM0rGdvLw7BOiz7faT-TiNPUQSw__&Key-Pair-Id=APKAJ67PQLWGCSP66DGA
Server: nginx/1.12.1
Strict-Transport-Security: max-age=63072000; preload
X-Frame-Options: DENY
$ http --header "https://cdn.quay.io/sha256/55/55f96fcd2154dea0366210d95996aa7f17c3a1d5a8ff86da075c038b56fe17b5?Expires=1590492686&Signature=MttvuW56~Obzu-t5lz6w8Xx8BFYMeWX3NekRdNHm0aGS1dncw7lBo5uzUB1LKtXxWM3gsK31-StT56rp9N05cKCzvNpw1oYrPslLdiLBBErexw~DJwYrDENheza-2AfO045IN7PS20NejpTTece55srCkXBRjZvdMM4N5AKSkFfVORST4AfPjaWjQenB9wyHQgca9Ux3EmK2UsYAhb953rjBA5sdo6MZV-I87CYHwCggsOwioG33PuGZV5LhgXGpjOUeA1QQPBE1a3M-MsvQ5bvvsWMinYRA7nPsnhyihMfomrL2VtTTyG7foeLAM0rGdvLw7BOiz7faT-TiNPUQSw__&Key-Pair-Id=APKAJ67PQLWGCSP66DGA"
HTTP/1.1 200 OK
Accept-Ranges: bytes
Connection: keep-alive
Content-Length: 967332469
Content-Type: binary/octet-stream
Date: Tue, 26 May 2020 11:22:57 GMT
ETag: "1f0c077b3d78c90bcb5f626336fa1f0f-1"
Last-Modified: Tue, 26 May 2020 09:49:00 GMT
Server: AmazonS3
Via: 1.1 77d8cf253666facea1bbe67902fcbbc1.cloudfront.net (CloudFront)
X-Amz-Cf-Id: p9yL-6-NU_zyRfW6Q_6RCtir33llG4e2HyfIx3OK4cPdQvQ9TGymZA==
X-Amz-Cf-Pop: DUS51-C1
X-Cache: Miss from cloudfront
x-amz-replication-status: COMPLETED
x-amz-server-side-encryption: AES256
x-amz-version-id: kl.cHTDXPhTMMv50HTFct7Z7VazmSMG7
Updated by ipanova@redhat.com over 4 years ago
I also have noticed that when fetching manfest a 200 is given and not 302 (GET/HEAD)
$ curl -i -X HEAD https://quay.io/v2/pulp/pulp-fixtures/manifests/latest -H "Accept: application/vnd.docker.distribution.manifest.v2+json"
HTTP/1.1 200 OK
Server: nginx/1.12.1
Date: Tue, 26 May 2020 12:29:42 GMT
Content-Type: application/vnd.docker.distribution.manifest.v2+json
Content-Length: 1784
Connection: close
Docker-Content-Digest: sha256:90c026a04b412530e76d96e3ec57f79775275b5ef2643cac8e867b111f739d82
X-Frame-Options: DENY
Strict-Transport-Security: max-age=63072000; preload
(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 ~]$ curl -i -X HEAD https://quay.io/v2/pulp/pulp-fixtures/manifests/sha256:90c026a04b412530e76d96e3ec57f79775275b5ef2643cac8e867b111f739d82 -H "Accept: application/vnd.docker.distribution.manifest.v2+json"
HTTP/1.1 200 OK
Server: nginx/1.12.1
Date: Tue, 26 May 2020 12:29:59 GMT
Content-Type: application/vnd.docker.distribution.manifest.v2+json
Content-Length: 1784
Connection: close
Docker-Content-Digest: sha256:90c026a04b412530e76d96e3ec57f79775275b5ef2643cac8e867b111f739d82
X-Frame-Options: DENY
Strict-Transport-Security: max-age=63072000; preload
$ curl -i -X GET https://quay.io/v2/pulp/pulp-fixtures/manifests/latest -H "Accept: application/vnd.docker.distribution.manifest.v2+json"
HTTP/1.1 200 OK
Server: nginx/1.12.1
Date: Tue, 26 May 2020 15:19:16 GMT
Content-Type: application/vnd.docker.distribution.manifest.v2+json
Content-Length: 1784
Connection: close
Docker-Content-Digest: sha256:90c026a04b412530e76d96e3ec57f79775275b5ef2643cac8e867b111f739d82
X-Frame-Options: DENY
Strict-Transport-Security: max-age=63072000; preload
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 7848,
"digest": "sha256:8e8b4fd66a958bbaa0be54c73b746bdb84b615982445713ff8362a35d11c7db2"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 27098756,
"digest": "sha256:afb6ec6fdc1c3ba04f7a56db32c5ff5ff38962dc4cd0ffdef5beaa0ce2eb77e2"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 23922586,
"digest": "sha256:b90c53a0b69244e37b3f8672579fc3dec13293eeb574fa0fdddf02da1e192fd6"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 203,
"digest": "sha256:11fa52a0fdc084d7fc3bbcb774389fd37b148ee98e7829cea4af189735acf848"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 216,
"digest": "sha256:12d9697316a6aeb3945a585c945100693cfa06a7019062418eface3c8722c800"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 967332469,
"digest": "sha256:55f96fcd2154dea0366210d95996aa7f17c3a1d5a8ff86da075c038b56fe17b5"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 69843,
"digest": "sha256:5d7549be9defb9c846d0b0bdf419adf7b917f261f82ced6d746d0b6786503e24"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 643,
"digest": "sha256:6557d2446dac29c7fca3113ca8690d5b17af292d07b9d8a7ccc92b3e15a2d9c2"
}
]
And GET/HEAD calls for blobs
http --header GET https://quay.io/v2/pulp/pulp-fixtures/blobs/sha256:55f96fcd2154dea0366210d95996aa7f17c3a1d5a8ff86da075c038b56fe17b5
HTTP/1.1 302 FOUND
Accept-Ranges: bytes
Cache-Control: max-age=31536000
Connection: close
Content-Length: 1225
Content-Type: text/html; charset=utf-8
Date: Tue, 26 May 2020 15:16:38 GMT
Docker-Content-Digest: sha256:55f96fcd2154dea0366210d95996aa7f17c3a1d5a8ff86da075c038b56fe17b5
Location: https://cdn.quay.io/sha256/55/55f96fcd2154dea0366210d95996aa7f17c3a1d5a8ff86da075c038b56fe17b5?Expires=1590506798&Signature=LSMOGnm54aneTVgXbWfuPJEhFSDJKBdHgRs8cubnwChsi~9C6Je-wI18thPlb6j3d2RrVzcsJiDSweTSa~Ni2MJ2MfqnujDf~lT9dfBSMHjmfTPdVhql6YvImd9pL5CvRuxD6dvFIpmA2Rs0H-Mwi9w6ud7TWdZZ5trkoA-vvrQtK-GQ95TktgGhdUszqUyo6wSQHrgW1WISeOveddHgrKmbYJDNX~rQBLVjNZYFKBoyO-oHTgmgNID73xTXdpHu54VqK0A9FVsaCdRsafygU52W4QhknRM7Pr77IwbR9zPNWCHDw7pnr22NR7aVhtpE9Tma4Q5VrTSsjb0ZO9ra8w__&Key-Pair-Id=APKAJ67PQLWGCSP66DGA
Server: nginx/1.12.1
Strict-Transport-Security: max-age=63072000; preload
X-Frame-Options: DENY
(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 ~]$ http --header HEAD https://quay.io/v2/pulp/pulp-fixtures/blobs/sha256:55f96fcd2154dea0366210d95996aa7f17c3a1d5a8ff86da075c038b56fe17b5
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: max-age=31436000
Connection: close
Content-Length: 967332469
Content-Type: application/octet-stream
Date: Tue, 26 May 2020 15:16:50 GMT
Docker-Content-Digest: sha256:55f96fcd2154dea0366210d95996aa7f17c3a1d5a8ff86da075c038b56fe17b5
Server: nginx/1.12.1
Strict-Transport-Security: max-age=63072000; preload
X-Frame-Options: DENY
Updated by mdellweg over 4 years ago
You just reminded me about what i really wanted to try. If i "HEAD" the blob instead of "GET" i also do not get the redirect. Can you verify that for the Manifests?
Updated by ipanova@redhat.com over 4 years ago
i believe it is enough to have set docker-content-digest for manifest only and not for blobs
This is dockerhub and headers it returns for blobs
$ ./docker-token library/busybox@sha256:d9cbbca60e5f0fc028b13c01e70e96cc2eefe7858ccc5a33c9a8faa8b03286aa
{'Date': 'Tue, 26 May 2020 12:39:13 GMT', 'Content-Type': 'application/octet-stream', 'Content-Length': '760847', 'Connection': 'keep-alive', 'Set-Cookie': '__cfduid=de5de30960a2908ef16c99893331488441590496753; expires=Thu, 25-Jun-20 12:39:13 GMT; path=/; domain=.production.cloudflare.docker.com; HttpOnly; SameSite=Lax; Secure', 'CF-Ray': '59978ec36c57f9e2-PRG', 'Accept-Ranges': 'bytes', 'Age': '1093147', 'Cache-Control': 'public, max-age=14400', 'ETag': '"17d26c5e6d0b638492f33daafaeb7d3b"', 'Expires': 'Tue, 26 May 2020 16:39:13 GMT', 'Last-Modified': 'Wed, 13 May 2020 20:48:14 GMT', 'Vary': 'Accept-Encoding', 'CF-Cache-Status': 'HIT', 'cf-request-id': '02f2978e210000f9e2d2378200000001', 'Expect-CT': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"', 'x-amz-id-2': 'TcdIEav8uoiC/qt6t6DXRWJZuy6KFhOUy9e5VedCjLn2JvXNYLFa5tL4sjTLTeHfw++fT8zmrWs=', 'x-amz-request-id': 'CA2C4EE257F33D9E', 'x-amz-version-id': 'NINdcIJ9IDpfyrDrWtGoiEcTjqEnomF1', 'Server': 'cloudflare'}
Updated by ipanova@redhat.com over 4 years ago
- Status changed from ASSIGNED to MODIFIED
Updated by mdellweg over 4 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE