Issue #5161
closed
Removing manifest_lists from a repository does not purge all newly unlinked manifests
Description
To reproduce:
1. Sync busybox
2. Create a new repo foo
3. add all manifest_lists from busybox to foo
4. remove all manifest lists from foo
5. check content unit counts, some manifests (and their referenced blobs) are still in foo
The problem occurs when removing 2 or more manifest_lists at the same time that reference the same manifests.
Manifest List A:
Manifest 1, 2, 3
Manifest LIst B:
Manifest 2, 3, 4
After adding AB then removing AB, Manifests 2, 3 will remain in the repo.
If removing the manifest_lists 1 at a time, no manifests remain in the repo (expected).
The faulty query is here:
https://github.com/pulp/pulp_docker/blob/2-master/plugins/pulp_docker/plugins/importers/importer.py#L501-L502
Unit filter should actually be:
unit_filters={'digest': {'$nin': [list, of, all, manifest_lists, being, removed]}}
This problem was discovered as a discrepency between a fix for https://pulp.plan.io/issues/4549 and the code on 2-master, and will be fixed in the same commit.
Related issues
.
Flatten queries for content unit removal
https://pulp.plan.io/issues/4549
Replace the recursive pattern with a fixed number of larger queries.
Additionally, reorder the content removal to "top down". This will fail more gracefully; failure leaves orphans (safe) rather than user-facing unlinked content (unsafe). This requires the additional plugin step of removing the explicitly given content, which is normally handled by pulp platform.
A side effect of this change is the correction of a bug that did not remove shared content, even if all linked content is removed. https://pulp.plan.io/issues/5161
fixes #5161 fixes #4549