Project

Profile

Help

Test #5181

Updated by bherring over 2 years ago

h3. ## Original Dev Problem

Removing all docker manifests from a large docker repo seems to take a long time:

~300 manifests takes ~2 minutes
~2000 manifests takes ~30-40 minutes

h3. Reproducer:

1. Create and sync a docker repo such as: https://quay.io datawire/ambassador
2. Remove all docker manifests from the repo: pulp-admin docker repo remove manifest --repo-id=1-docker-dev-7915f7d0-7a98-4131-9c41-1be7b578d442 --not id=foo

h3. ## Discussion with asmacado

* A lot of the delay were caused by the sequential removals
* If there was an outage while the task was on-going, the removal was left in a non-deterministic state
* The timing model for removal for general use cases grew incredibly fast

h3. Solutions with the new refactor:

* All BLOBS and MANIFESTS are all removed at once
* Higher level units are all removed second
* Timing shrank from 300 ~=1hr to 300 ~=a few seconds

h3. Concerns:


* This refactor would affect removals and syncs
* The removal, if interrupted, would leave Orphans which could affect upstream verification verifications

h3. Actions:

h4. Upstream

* Need an upstream verification that ensures there is no change in functionality with remove units and sync units

h4. Downstream


* Need to verify downstream verification verifications are not affected by this change/patch
* BZ verification will just be the removal of a large number of repos takes a short amount of time

Back