Pulp

Question 1: which endpoint?¶

I was wondering if this should be an action endpoint or not underneath sync, e.g. /fix/ or /repair/. I decided it should not be because:

a) we may want to offer this functionality system-wide later
b) plugin writers would have to enable that endpoint. We could enable it for all plugins without their involvement, but that weakens the practice that repo detail endpoints are plugin owned.

So if we did it system wide would it be?

/pulp/api/v3/repair/
/pulp/api/v3/filesystem-repair/
/pulp/api/v3/repo-repair/
/pulp/api/v3/fix/
/pulp/api/v3/filesystem-fix/
/pulp/api/v3/repo-fix/

Question 2: What Pulp locks need to guard this task?¶

I think we need to lock on the repository so that other repository operations won't make new repo versions underneath the content while it's being examined. System-wide checking (not proposed in this ticket) would be a whole different story.

Question 3: To confirm, this only fixes "downloaded content" right?¶

So if we implement a "continue even if 404s occur" and say two RPMs are missing from the latest repo version. This utility won't also fix that. If it's expected to also fix that then we need to integrate this deeply with sync also somehow. What can we do that is the best regarding this case?

Couple of comments:

Question 2: This makes sense to me

Question 3:

"So if we implement a "continue even if 404s occur" and say two RPMs are missing from the latest repo version. This utility won't also fix that. If it's expected to also fix that then we need to integrate this deeply with sync also somehow. What can we do that is the best regarding this case?"

I think this repair task not fixing is fine, since it was never downloaded in the first place. But what would fix that? A further sync? (assuming the underlying upstream repository is fixed too).

jsherril@redhat.com wrote:

I think this repair task not fixing is fine, since it was never downloaded in the first place. But what would fix that? A further sync? (assuming the underlying upstream repository is fixed too).

Yes a further sync would resolve this if the upstream repository was also fixed. Pulp is going to recognize it does not have the content unit during that sync.

Question 3:

"So if we implement a "continue even if 404s occur" and say two RPMs are missing from the latest repo version. This utility won't also fix that. If it's expected to also fix that then we need to integrate this deeply with sync also somehow. What can we do that is the best regarding this case?"

I think this repair task not fixing is fine, since it was never downloaded in the first place. But what would fix that? A further sync? (assuming the underlying upstream repository is fixed too).

For the upcoming sync to fix it, a sync should be always operational, with no condition to skip it.
How confident are we that every sync being operational is good enough performance-wise? If nothing changed upstream, we'll still analyse all the repodata, we'll check if every content exists in pulp and then we may perform all the checks to ensure that repo is valid (I'm not sure if we are performing this check if nothing changed in a repo). Maybe it's fast and no need to be worried.

If we want any optimization, we'll either need to introduce force_full to pulp3, or we may need to do full repair here and not only downloaded content.
What do you think?

Also we potentially can have different options for the suggested endpoints:
/pulp/api/v3/repair/ - full repair = corrupted files + download missing ones
/pulp/api/v3/filesystem-repair/ - only corrupted files
/pulp/api/v3/repo-repair/ - corrupted and downloaded for a specific repo

I'm worried that having all those options adds complexity but at the same time it gives flexibility.

ttereshc wrote:

For the upcoming sync to fix it, a sync should be always operational, with no condition to skip it.

Agreed

How confident are we that every sync being operational is good enough performance-wise? If nothing changed upstream, we'll still analyse all the repodata, we'll check if every content exists in pulp and then we may perform all the checks to ensure that repo is valid (I'm not sure if we are performing this check if nothing changed in a repo). Maybe it's fast and no need to be worried.

I suspect no matter how fast it is, users will want it to be faster in cases when it can be. I'm interested in us measuring the resync time in the case that nothing changed for RPM first so we can understand how much opportunity for speedup there is.

If we want any optimization, we'll either need to introduce force_full to pulp3, or we may need to do full repair here and not only downloaded content.
What do you think?

What I learned from pulp2 is that anytime we introduce an optomization we need a way to turn it (or all of them) off.

Also we potentially can have different options for the suggested endpoints:
/pulp/api/v3/repair/ - full repair = corrupted files + download missing ones
/pulp/api/v3/filesystem-repair/ - only corrupted files
/pulp/api/v3/repo-repair/ - corrupted and downloaded for a specific repo

I'm worried that having all those options adds complexity but at the same time it gives flexibility.

This is my primary concern also.

That looks good to me!

FWIW, in RPM plugin we have sync optimizations, so if nothing changed in a remote repo and no changes have been made to a pulp repo, then the upcoming sync will be a no-op one. I think it's ok, it's a rare case when one hits 404 for content download and then it started working with no changes.

New response format looks good to me. +1

I am in the process of creating a POC with a slightly different design. As the operation is really repairing the artifacts in a repository version, i am adding a task that is triggered by detail endpoint of repository_version with the name 'repair'. To support the repair a list of repositories workflow, a second endpoint can be added to trigger a task with subtasks of the above type.

The python bindings to access the nested endpoint look like: pulpcore.client.pulp_file.api.repositories_file_versions_api.RepositoriesFileVersionsApi.repair