Test #5181
Updated by bherring over 4 years ago
h3. Original Dev Problem
Removing all docker manifests from a large docker repo seems to take a long time:
~300 manifests takes ~2 minutes
~2000 manifests takes ~30-40 minutes
h3. Reproducer:
1. Create and sync a docker repo such as: https://quay.io datawire/ambassador
2. Remove all docker manifests from the repo: pulp-admin docker repo remove manifest --repo-id=1-docker-dev-7915f7d0-7a98-4131-9c41-1be7b578d442 --not id=foo
h3. Discussion with asmacado
* A lot of the delay were caused by the sequential removals
* If there was an outage while the task was on-going, the removal was left in a non-deterministic state
* The timing model for removal for general use cases grew incredibly fast
h3. Solutions with the new refactor:
* All BLOBS and MANIFESTS are all removed at once
* Higher level units are all removed second
* Timing shrank from 300 ~=1hr to 300 ~=a few seconds
h3. Concerns:
* This refactor would affect removals and syncs
* The removal, if interrupted, would leave Orphans which could affect upstream verification
h3. Actions:
h4. Upstream
* Need an upstream verification that ensures there is no change in functionality with remove units and sync units
h4. Downstream
* Need to verify downstream verification are not affected by this change/patch
* BZ verification will just be the removal of a large number of repos takes a short amount of time
h3. Test Permutations
h4. Remove
Variables:
* (Permutations) Shared or Not Shared [manifest_lists, manifests]
* type_ids": ['docker_tag', 'docker_manifest_list', 'docker_manifest', 'docker_blob']
Verification Scenarios:
* For Not Shared, remove $REMOVAL_TYPES and verify all units have been removed
* For Shared, remove $REMOVAL_TYPES and verify only non-shared units have been removed. Shared units should remain.
Note:
* Removal Criteria short-cut for relevant type_id being removed.
<pre>
criteria = {
"type_ids": ['docker_tag', 'docker_manifest_list', 'docker_manifest', 'docker_blob'],
"filters": {"unit": {"_id": {"$in": all_units}}},
}
</pre>
* There are already tests covering tag removal
.