Project

Profile

Help

Story #3450

closed

As a user I would like to limit the tags we sync for docker repos

Added by jmontleo@redhat.com about 6 years ago. Updated over 2 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.17.0
Target Release - Docker:
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
Sprint 38
Quarter:

Description

Motivation:

We're currently sync'ing some very large openshift repos from brew. In reality we're only interested in the current 3.9 and newerreleases/tags but we're getting everything, which is weighing in at 1.2TB at present and growing with each puddle for each version.

It would be nice to be able to limit the tags we sync.

Solution:

Provide a config on the DockerImporter config called "tags". This config will accept a list of tags to sync.

Provide a CLI option called "--tags". The user will be able to specify "--tags" when creating a docker repository. The user will also be able to override the "--tags" importer config at sync time, by specifying it as part of the override_configs.

When the "tags" option is provided:

Pulp will sync just those tags + manifests associated with the tags and all the corresponding layers. Sync will only be performed using the immediate download policy. There will be no support for the on_demand download policy.

Note: If the list of tags was changed, and tag X is not wanted anymore, a manual removal of that tag X should occur in the local repo.
Note: If some invalid tags provided are invalid, Pulp will only sync the available ones in the remote repo.

When the "tags" option is not provided:
Pulp will sync the whole repo.


Related issues

Has duplicate Docker Support - Story #3623: As a user I can sync a registry by whitelisting tags ( Filtered Sync)CLOSED - DUPLICATE

Actions
Actions #1

Updated by tomckay@redhat.com about 6 years ago

This usage would also be useful with foreman as well. I can see two ways:
1. Provide a tag filter up front to limit tags pulled down
2. Just pull the metadata files for the tags and sync the actual images later (exactly like for RPM units)

Personally, I would prefer the second option.

Actions #2

Updated by ipanova@redhat.com almost 6 years ago

  • Has duplicate Story #3137: As a user, I can view docker image Arch and Size Information through the REST API added
Actions #3

Updated by ipanova@redhat.com almost 6 years ago

  • Has duplicate deleted (Story #3137: As a user, I can view docker image Arch and Size Information through the REST API)
Actions #4

Updated by ipanova@redhat.com almost 6 years ago

  • Has duplicate Story #3623: As a user I can sync a registry by whitelisting tags ( Filtered Sync) added
Actions #5

Updated by ipanova@redhat.com almost 6 years ago

an option will be provided where a list of tags to sync can be specified. for example --tags

if option is provided:
we will sync just those tags + its' manifests + correspondent layers. Sync is performed in immediate poilicy, no on_demand.
note if the list of tags was changed, and tag X is not wanted anymore, a manual removal of that tag X should occur in the local repo.
note if some of invalid tags where provided in the list to sync, we will sync just available ones in the remote repo.

if option is not provided:
we will sync by default whole repo.

Actions #6

Updated by ipanova@redhat.com almost 6 years ago

  • Sprint Candidate changed from No to Yes
Actions #7

Updated by dkliban@redhat.com almost 6 years ago

  • Description updated (diff)
Actions #8

Updated by ipanova@redhat.com almost 6 years ago

  • Groomed changed from No to Yes
Actions #9

Updated by rchan almost 6 years ago

  • Sprint set to Sprint 38
Actions #10

Updated by bizhang almost 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to bizhang
Actions #11

Updated by bizhang almost 6 years ago

  • Status changed from ASSIGNED to POST

Added by werwty almost 6 years ago

Revision e46d3b3e | View on GitHub

Add feature to whitelist tags on sync.

closes #3450 https://pulp.plan.io/issues/3450

Added by werwty almost 6 years ago

Revision e46d3b3e | View on GitHub

Add feature to whitelist tags on sync.

closes #3450 https://pulp.plan.io/issues/3450

Actions #12

Updated by werwty almost 6 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100
Actions #13

Updated by dkliban@redhat.com almost 6 years ago

  • Platform Release set to 2.16.2
Actions #14

Updated by dkliban@redhat.com almost 6 years ago

  • Platform Release deleted (2.16.2)
Actions #15

Updated by amacdona@redhat.com almost 6 years ago

  • Platform Release set to 2.16.2
Actions #16

Updated by amacdona@redhat.com almost 6 years ago

  • Platform Release deleted (2.16.2)
Actions #17

Updated by ipanova@redhat.com almost 6 years ago

  • Platform Release set to 2.17.0
Actions #18

Updated by rchan almost 6 years ago

  • Sprint/Milestone set to 2.17.0

Adding to 2.17.0 milestone. This is one of the required deliverables.

Actions #19

Updated by ipanova@redhat.com over 5 years ago

  • Platform Release changed from 2.17.0 to master
Actions #20

Updated by ipanova@redhat.com over 5 years ago

  • Platform Release deleted (master)
Actions #21

Updated by ipanova@redhat.com over 5 years ago

  • Platform Release set to 2.17.0
Actions #22

Updated by ipanova@redhat.com over 5 years ago

  • Status changed from MODIFIED to 5
Actions #23

Updated by ipanova@redhat.com over 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #25

Updated by bmbouter about 5 years ago

  • Tags Pulp 2 added
Actions #26

Updated by ttereshc over 2 years ago

  • Sprint/Milestone deleted (2.17.0)

Also available in: Atom PDF