Actions

Send by e-mail Copy link

Story #8459

closed

As a user I want to reclaim disk space for a list of repositories

Added by ipanova@redhat.com over 3 years ago. Updated over 3 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Normal

Assignee:

ipanova@redhat.com

Category:

Sprint/Milestone:

3.15.0

Start date:

Due date:

% Done:

100%

Estimated time:

Platform Release:

Groomed:

Yes

Sprint Candidate:

Yes

Tags:

GalaxyNG, Katello

Sprint:

Sprint 102

Quarter:

Description

Problem Statement/Use cases:¶

As a user I might have content that I no longer need to serve but I want to keep it in the repo for history purposes. I want to still be able to free up some disk space in such case.
As a user I have repos that have on_demand download policy and I want to clear out downloaded files for those repos. A repository with on_demand download policy will store artifacts locally after they have been requested, but there really isn't a way to have pulp delete the locally stored files and free disk space if these packages are unlikely to be used again.

Solution¶

pulp/api/v3/repositories/<reclaim-disk-space>/ Provide a separate endpoint which will accept a list of repo hrefs to reclaim disk space for.

Artifacts only exclusively to that list of repos will be removed. No new versions will be created or removed because content set will not change.
a keeplist option will contain list of repo_version hrefs which will be excluded from Artifact removal.
The task will only remove those Artifacts that have a corresponding Remote artifact.
There, however, will be a force flag that will remove Artifacts that have no RA( aka uploaded content). This will not be a default behavior.
The task will trim Artifacts regardless of the download policy. ( Content app would be able to stream artifact if it is locally available otherwise with the help of RA( if available) it would download artifact from the remote source)

https://hackmd.io/e78_-T2rQRWvXaGjHT6Kyw

Related issues

Actions

Copy link

Updated by ipanova@redhat.com over 3 years ago

Description updated (diff)

Actions

Copy link

Updated by ipanova@redhat.com over 3 years ago

Description updated (diff)

Actions

Copy link

Updated by ttereshc over 3 years ago

I suggest to change the language from deferred download policy to on_demand download policy.

Other than that it looks good and +1 to keeplist or some kind of labeling to preserve repo versions of interest. If a user would want to reclaim disc space on a regular basis, it might not be convenient to provide the keeplist each time, so maybe labeling repo versions in some way would be a better approach.

Actions

Copy link

Updated by mdellweg over 3 years ago

This is kind of the anti-feature of the repoversion-repair. Maybe it's wise to try to give them similar apis. I think that would mean the atom to work on would not be the repository, but the repository_version. You'd probably need to pass a list of versions to get a meaningful feature here. But it would give you the keep_list functionality for free (Just don't specify the ones to preserve).

Actions

Copy link

Updated by ipanova@redhat.com over 3 years ago

ttereshc wrote:

I suggest to change the language from deferred download policy to on_demand download policy.

Other than that it looks good and +1 to keeplist or some kind of labeling to preserve repo versions of interest. If a user would want to reclaim disc space on a regular basis, it might not be convenient to provide the keeplist each time, so maybe labeling repo versions in some way would be a better approach.

To my knowledge labeling is available only on the repos and not repo versions. I agree that labels are more user friendly

Actions

Copy link

Updated by ipanova@redhat.com over 3 years ago

Description updated (diff)

Actions

Copy link

Updated by ipanova@redhat.com over 3 years ago

Groomed changed from No to Yes
Sprint Candidate changed from No to Yes

Actions

Copy link

Updated by newswangerd over 3 years ago

Will there be some other mechanism to do this, but for a specific set of content in a repository? We'd also like to be able to clean up a list of content in a repository without completely wiping out all content in previous versions of a repo.

Actions

Copy link

#10

Updated by newswangerd over 3 years ago

I guess we can accomplish this by removing the content from the latest version of the repo and then run the task to purge content on all versions except the latest.

Actions

Copy link

#11

Updated by ipanova@redhat.com over 3 years ago

Tags GalaxyNG added

Actions

Copy link

#12

Updated by ipanova@redhat.com over 3 years ago

Tags Katello added

Actions

Copy link

#13

Updated by ipanova@redhat.com over 3 years ago

Sprint/Milestone set to Content/disk space management

Actions

Copy link

#14

Updated by ttereshc over 3 years ago

Should this #5926 be closed as a dupe of this one?

Actions

Copy link

#15

Updated by ipanova@redhat.com over 3 years ago

ttereshc wrote:

Should this #5926 be closed as a dupe of this one?

Yes this can closed. Thanks for the reminder.

Actions

Copy link

#16

Updated by ttereshc over 3 years ago

Has duplicate Story #5926: As a user, I can clear out downloaded files from on_demand repos added

Actions

Copy link

#17

Updated by ipanova@redhat.com over 3 years ago

Description updated (diff)

Actions

Copy link

#18

Updated by dalley over 3 years ago

When we do this we should possibly consider (or at least think about the use case) of doing the opposite. "download everything in this repository, even if it came from multiple remotes originally"

Actions

Copy link

#19