Project

Profile

Help

Story #3450

As a user I would like to limit the tags we sync for docker repos

Added by jmontleo@redhat.com over 1 year ago. Updated 2 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
% Done:

100%

Platform Release:
2.17.0
Blocks Release:
Target Release - Docker:
Backwards Incompatible:
No
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 38

Description

Motivation:

We're currently sync'ing some very large openshift repos from brew. In reality we're only interested in the current 3.9 and newerreleases/tags but we're getting everything, which is weighing in at 1.2TB at present and growing with each puddle for each version.

It would be nice to be able to limit the tags we sync.

Solution:

Provide a config on the DockerImporter config called "tags". This config will accept a list of tags to sync.

Provide a CLI option called "--tags". The user will be able to specify "--tags" when creating a docker repository. The user will also be able to override the "--tags" importer config at sync time, by specifying it as part of the override_configs.

When the "tags" option is provided:

Pulp will sync just those tags + manifests associated with the tags and all the corresponding layers. Sync will only be performed using the immediate download policy. There will be no support for the on_demand download policy.

Note: If the list of tags was changed, and tag X is not wanted anymore, a manual removal of that tag X should occur in the local repo.
Note: If some invalid tags provided are invalid, Pulp will only sync the available ones in the remote repo.

When the "tags" option is not provided:

Pulp will sync the whole repo.


Checklist


Related issues

Duplicated by Docker Support - Story #3623: As a user I can sync a registry by whitelisting tags ( Filtered Sync) CLOSED - DUPLICATE Actions

Associated revisions

Revision e46d3b3e View on GitHub
Added by werwty about 1 year ago

Add feature to whitelist tags on sync.

closes #3450
https://pulp.plan.io/issues/3450

Revision e46d3b3e View on GitHub
Added by werwty about 1 year ago

Add feature to whitelist tags on sync.

closes #3450
https://pulp.plan.io/issues/3450

History

#1 Updated by tomckay@redhat.com over 1 year ago

This usage would also be useful with foreman as well. I can see two ways:
1. Provide a tag filter up front to limit tags pulled down
2. Just pull the metadata files for the tags and sync the actual images later (exactly like for RPM units)

Personally, I would prefer the second option.

#2 Updated by ipanova@redhat.com about 1 year ago

  • Duplicated by Story #3137: As a user, I can view docker image Arch and Size Information through the REST API added

#3 Updated by ipanova@redhat.com about 1 year ago

  • Duplicated by deleted (Story #3137: As a user, I can view docker image Arch and Size Information through the REST API)

#4 Updated by ipanova@redhat.com about 1 year ago

  • Duplicated by Story #3623: As a user I can sync a registry by whitelisting tags ( Filtered Sync) added

#5 Updated by ipanova@redhat.com about 1 year ago

an option will be provided where a list of tags to sync can be specified. for example --tags

if option is provided:
we will sync just those tags + its' manifests + correspondent layers. Sync is performed in immediate poilicy, no on_demand.
note if the list of tags was changed, and tag X is not wanted anymore, a manual removal of that tag X should occur in the local repo.
note if some of invalid tags where provided in the list to sync, we will sync just available ones in the remote repo.

if option is not provided:
we will sync by default whole repo.

#6 Updated by ipanova@redhat.com about 1 year ago

  • Sprint Candidate changed from No to Yes

#7 Updated by dkliban@redhat.com about 1 year ago

  • Description updated (diff)

#8 Updated by ipanova@redhat.com about 1 year ago

  • Groomed changed from No to Yes

#9 Updated by rchan about 1 year ago

  • Sprint set to Sprint 38

#10 Updated by bizhang about 1 year ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to bizhang

#11 Updated by bizhang about 1 year ago

  • Status changed from ASSIGNED to POST
  • Smash Test set to 1064

#12 Updated by werwty about 1 year ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#13 Updated by dkliban@redhat.com about 1 year ago

  • Platform Release set to 2.16.2

#14 Updated by dkliban@redhat.com about 1 year ago

  • Platform Release deleted (2.16.2)

#15 Updated by amacdona@redhat.com about 1 year ago

  • Platform Release set to 2.16.2

#16 Updated by amacdona@redhat.com about 1 year ago

  • Platform Release deleted (2.16.2)

#17 Updated by ipanova@redhat.com 12 months ago

  • Platform Release set to 2.17.0

#18 Updated by rchan 12 months ago

  • Sprint/Milestone set to 2.17.0

Adding to 2.17.0 milestone. This is one of the required deliverables.

#19 Updated by ipanova@redhat.com 11 months ago

  • Platform Release changed from 2.17.0 to master

#20 Updated by ipanova@redhat.com 11 months ago

  • Platform Release deleted (master)

#21 Updated by ipanova@redhat.com 11 months ago

  • Platform Release set to 2.17.0

#22 Updated by ipanova@redhat.com 10 months ago

  • Status changed from MODIFIED to ON_QA

#23 Updated by ipanova@redhat.com 10 months ago

  • Status changed from ON_QA to CLOSED - CURRENTRELEASE

#25 Updated by bmbouter 2 months ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF