Project

Profile

Help

Story #8459

As a user I want to reclaim disk space for a list of repositories

Added by ipanova@redhat.com 7 months ago. Updated about 2 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
GalaxyNG, Katello
Sprint:
Sprint 102
Quarter:

Description

Problem Statement/Use cases:

  • As a user I might have content that I no longer need to serve but I want to keep it in the repo for history purposes. I want to still be able to free up some disk space in such case.
  • As a user I have repos that have on_demand download policy and I want to clear out downloaded files for those repos. A repository with on_demand download policy will store artifacts locally after they have been requested, but there really isn't a way to have pulp delete the locally stored files and free disk space if these packages are unlikely to be used again.

Solution

pulp/api/v3/repositories/<reclaim-disk-space>/ Provide a separate endpoint which will accept a list of repo hrefs to reclaim disk space for.

  • Artifacts only exclusively to that list of repos will be removed. No new versions will be created or removed because content set will not change.
  • a keeplist option will contain list of repo_version hrefs which will be excluded from Artifact removal.
  • The task will only remove those Artifacts that have a corresponding Remote artifact.
  • There, however, will be a force flag that will remove Artifacts that have no RA( aka uploaded content). This will not be a default behavior.
  • The task will trim Artifacts regardless of the download policy. ( Content app would be able to stream artifact if it is locally available otherwise with the help of RA( if available) it would download artifact from the remote source)

https://hackmd.io/e78_-T2rQRWvXaGjHT6Kyw


Related issues

Related to Pulp - Story #8324: As a user, I can have Pulp download content for a repository versionNEW

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>
Has duplicate Pulp - Story #5926: As a user, I can clear out downloaded files from on_demand reposCLOSED - DUPLICATE

<a title="Actions" class="icon-only icon-actions js-contextmenu" href="#">Actions</a>

Associated revisions

History

#1 Updated by ipanova@redhat.com 7 months ago

  • Description updated (diff)

#2 Updated by ipanova@redhat.com 7 months ago

  • Description updated (diff)

#4 Updated by ttereshc 6 months ago

I suggest to change the language from deferred download policy to on_demand download policy.

Other than that it looks good and +1 to keeplist or some kind of labeling to preserve repo versions of interest. If a user would want to reclaim disc space on a regular basis, it might not be convenient to provide the keeplist each time, so maybe labeling repo versions in some way would be a better approach.

#5 Updated by mdellweg 6 months ago

This is kind of the anti-feature of the repoversion-repair. Maybe it's wise to try to give them similar apis. I think that would mean the atom to work on would not be the repository, but the repository_version. You'd probably need to pass a list of versions to get a meaningful feature here. But it would give you the keep_list functionality for free (Just don't specify the ones to preserve).

#6 Updated by ipanova@redhat.com 6 months ago

ttereshc wrote:

I suggest to change the language from deferred download policy to on_demand download policy.

Other than that it looks good and +1 to keeplist or some kind of labeling to preserve repo versions of interest. If a user would want to reclaim disc space on a regular basis, it might not be convenient to provide the keeplist each time, so maybe labeling repo versions in some way would be a better approach.

To my knowledge labeling is available only on the repos and not repo versions. I agree that labels are more user friendly

#7 Updated by ipanova@redhat.com 6 months ago

  • Description updated (diff)

#8 Updated by ipanova@redhat.com 6 months ago

  • Groomed changed from No to Yes
  • Sprint Candidate changed from No to Yes

#9 Updated by newswangerd 6 months ago

Will there be some other mechanism to do this, but for a specific set of content in a repository? We'd also like to be able to clean up a list of content in a repository without completely wiping out all content in previous versions of a repo.

#10 Updated by newswangerd 6 months ago

I guess we can accomplish this by removing the content from the latest version of the repo and then run the task to purge content on all versions except the latest.

#11 Updated by ipanova@redhat.com 5 months ago

  • Tags GalaxyNG added

#12 Updated by ipanova@redhat.com 5 months ago

  • Tags Katello added

#13 Updated by ipanova@redhat.com 4 months ago

  • Sprint/Milestone set to Content/disk space management

#14 Updated by ttereshc 4 months ago

Should this #5926 be closed as a dupe of this one?

#15 Updated by ipanova@redhat.com 4 months ago

ttereshc wrote:

Should this #5926 be closed as a dupe of this one?

Yes this can closed. Thanks for the reminder.

#16 Updated by ttereshc 4 months ago

  • Has duplicate Story #5926: As a user, I can clear out downloaded files from on_demand repos added

#17 Updated by ipanova@redhat.com 4 months ago

  • Description updated (diff)

#18 Updated by dalley 4 months ago

When we do this we should possibly consider (or at least think about the use case) of doing the opposite. "download everything in this repository, even if it came from multiple remotes originally"

#19 Updated by daviddavis 4 months ago

dalley I filed a feature request that I think is similar to your suggestion: https://pulp.plan.io/issues/8324

#20 Updated by daviddavis 4 months ago

  • Sprint/Milestone changed from Content/disk space management to 3.15.0

#21 Updated by dalley 4 months ago

  • Related to Story #8324: As a user, I can have Pulp download content for a repository version added

#22 Updated by ipanova@redhat.com 4 months ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to ipanova@redhat.com
  • Sprint set to Sprint 99

#23 Updated by pulpbot 4 months ago

  • Status changed from ASSIGNED to POST

#24 Updated by rchan 4 months ago

  • Sprint changed from Sprint 99 to Sprint 100

#25 Updated by rchan 3 months ago

  • Sprint changed from Sprint 100 to Sprint 101

#26 Updated by ipanova@redhat.com 2 months ago

  • Sprint changed from Sprint 101 to Sprint 102

#27 Updated by ipanova@redhat.com 2 months ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#28 Updated by pulpbot about 2 months ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE

Please register to edit this issue

Also available in: Atom PDF