Project

Profile

Help

Story #9241

closed

As a user I can mount blobs across repositories

Added by ipanova@redhat.com over 2 years ago. Updated over 2 years ago.

Status:
CLOSED - DUPLICATE
Priority:
Normal
Assignee:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Ticket moved to GitHub: "pulp/pulp_container/494":https://github.com/pulp/pulp_container/issues/494


Given that the user, who is currently pushing an image, has READ access to another repository that already contains the blob, he should be able to mount blobs across repositories. The ability, that registry may provide is beneficial to save on network speed.

Cross blob mount works in the following way: For every repository that container blob X will be asked a token with mount permission. If the user has no READ permission to any of those repos it will fall back and create the blob upload and upload the blob bits.

In these logs the are already existing repos busybox, busybox1, busybox2 that share same blob. Repo ina/busybox3 is being pushed and it has same blob that is contained in busybox, busybox1 and busybox2. It tries to mount blob against each repo, fails and falls back to the regular blob upload.

.0.0.1 - - [16/Aug/2021:14:37:08 +0000] "PUT /v2/ina/busybox3/blobs/uploads/5e6887df-78f7-4908-ab7d-1e03a97754fd?digest=sha256%3Aa9efad4c351baa14138b2ddca283c33974c1957b59981f77642ccf4398981045 HTTP/1.1" 201

.0.0.1 - - [16/Aug/2021:14:37:07 +0000] "PATCH /v2/ina/busybox3/blobs/uploads/5e6887df-78f7-4908-ab7d-1e03a97754fd HTTP/1.1" 202 
.0.0.1 - - [16/Aug/2021:14:37:07 +0000] "POST /v2/ina/busybox3/blobs/uploads/ HTTP/1.1" 202 

.0.0.1 - - [16/Aug/2021:14:37:06 +0000] "POST /v2/ina/busybox3/blobs/uploads/?from=busybox&mount=sha256%3Aa9efad4c351baa14138b2ddca283c33974c1957b59981f77642ccf4398981045 HTTP/1.1" 401 
ngo.request:WARNING: Unauthorized: /v2/ina/busybox3/blobs/uploads/
 /pulp/container/busybox/blobs/sha256:a9efad4c351baa14138b2ddca283c33974c1957b59981f77642ccf4398981045?validate_token=bb5a4a4fffc5418d6b8d8f735e49caf38bb4660cf68a4a6fca3bc6e196550e60:fb629a583d896ba3e1a4888a83153a733a85e677771c2fe5a27342>
.0.0.1 - - [16/Aug/2021:14:37:06 +0000] "HEAD /v2/busybox/blobs/sha256:a9efad4c351baa14138b2ddca283c33974c1957b59981f77642ccf4398981045 HTTP/1.1" 302 
.0.0.1 - ina [16/Aug/2021:14:37:06 +0000] "GET /token/?account=ina&scope=repository%3Aina%2Fbusybox3%3Apull%2Cpush&scope=repository%3Abusybox%3Apull&service=localhost%3A24817 HTTP/1.0" 200 
.0.0.1 - - [16/Aug/2021:14:37:06 +0000] "POST /v2/ina/busybox3/blobs/uploads/?from=busybox1&mount=sha256%3Aa9efad4c351baa14138b2ddca283c33974c1957b59981f77642ccf4398981045 HTTP/1.1" 401 
ngo.request:WARNING: Unauthorized: /v2/ina/busybox3/blobs/uploads/
 /pulp/container/busybox1/blobs/sha256:a9efad4c351baa14138b2ddca283c33974c1957b59981f77642ccf4398981045?validate_token=d38d17fd475a24e228ed9d0418b937d7011954227d42945d9f5c14cca4afa50c:23939ede17923e540b63991e7372e1abed2223a26497c0581531b>
.0.0.1 - - [16/Aug/2021:14:37:06 +0000] "HEAD /v2/busybox1/blobs/sha256:a9efad4c351baa14138b2ddca283c33974c1957b59981f77642ccf4398981045 HTTP/1.1" 302 
.0.0.1 - ina [16/Aug/2021:14:37:06 +0000] "GET /token/?account=ina&scope=repository%3Aina%2Fbusybox3%3Apull%2Cpush&scope=repository%3Abusybox1%3Apull&service=localhost%3A24817 HTTP/1.0" 200 
.0.0.1 - - [16/Aug/2021:14:37:06 +0000] "POST /v2/ina/busybox3/blobs/uploads/?from=busybox2&mount=sha256%3Aa9efad4c351baa14138b2ddca283c33974c1957b59981f77642ccf4398981045 HTTP/1.1" 401 
ngo.request:WARNING: Unauthorized: /v2/ina/busybox3/blobs/uploads/
 /pulp/container/busybox2/blobs/sha256:a9efad4c351baa14138b2ddca283c33974c1957b59981f77642ccf4398981045?validate_token=4148bac928c3c8641bf5a40839c4e082334fc5242a787f99117eb956d4d99906:59026249fe55842485fbecd279c8b98e33df47c168ecf02cd6df4>
.0.0.1 - - [16/Aug/2021:14:37:05 +0000] "HEAD /v2/busybox2/blobs/sha256:a9efad4c351baa14138b2ddca283c33974c1957b59981f77642ccf4398981045 HTTP/1.1" 302 
.0.0.1 - ina [16/Aug/2021:14:37:05 +0000] "GET /token/?account=ina&scope=repository%3Aina%2Fbusybox3%3Apull%2Cpush&scope=repository%3Abusybox2%3Apull&service=localhost%3A24817 HTTP/1.0" 200
.0.0.1 - - [16/Aug/2021:14:37:05 +0000] "HEAD /v2/ina/busybox3/blobs/sha256:5b8c72934dfc08c7d2bd707e93197550f06c0751023dabb3a045b723c5e7b373 HTTP/1.1" 404 
ngo.request:WARNING: Not Found: /v2/ina/busybox3/blobs/sha256:5b8c72934dfc08c7d2bd707e93197550f06c0751023dabb3a045b723c5e7b373
.0.0.1 - ina [16/Aug/2021:14:37:05 +0000] "GET /token/?account=ina&scope=repository%3Aina%2Fbusybox3%3Apull%2Cpush&service=localhost%3A24817 HTTP/1.0" 200 
.0.0.1 - - [16/Aug/2021:14:37:05 +0000] "GET /v2/ HTTP/1.1" 401 
ngo.request:WARNING: Unauthorized: /v2/

https://docs.docker.com/search/?q=Cross%20Repository%20Blob%20Mount

Actions #1

Updated by ipanova@redhat.com over 2 years ago

  • Description updated (diff)
Actions #2

Updated by ipanova@redhat.com over 2 years ago

  • Description updated (diff)
Actions #3

Updated by mgoddard over 2 years ago

Would love to see this!

We have many images which are part of a hierarchy, and have many shared layers. Currently pushing all of these to pulp takes over 3 hours since we have to push all layers for each image.

Actions #4

Updated by ipanova@redhat.com over 2 years ago

  • Sprint/Milestone set to 2.10.0
Actions #5

Updated by ipanova@redhat.com over 2 years ago

  • Sprint/Milestone changed from 2.10.0 to 2.11.0
Actions #6

Updated by pulpbot over 2 years ago

  • Description updated (diff)
  • Status changed from NEW to CLOSED - DUPLICATE

Also available in: Atom PDF