Test #5181

Updated by bherring almost 5 years ago

h3.    ## Original Dev Problem 

 Removing all docker manifests from a large docker repo seems to take a long time: 

 ~300 manifests takes ~2 minutes 
 ~2000 manifests takes ~30-40 minutes 

 h3.    Reproducer: 

 1.    Create and sync a docker repo such as:         datawire/ambassador 
 2.    Remove all docker manifests from the    repo:      pulp-admin    docker repo remove manifest --repo-id=1-docker-dev-7915f7d0-7a98-4131-9c41-1be7b578d442     --not id=foo 

 h3.    ## Discussion with asmacado 

 * A lot of the delay were caused by the sequential removals 
 * If there was an outage while the task was on-going, the removal was left in a non-deterministic state 
 * The timing model for removal for general use cases grew incredibly fast 

 h3.    Solutions with the new refactor: 

 * All BLOBS and MANIFESTS are all removed at once 
 * Higher level units are all removed second 
 * Timing shrank from 300 ~=1hr to 300 ~=a few seconds 

 h3. Concerns: 

 * This refactor would affect removals and syncs 
 * The removal, if interrupted, would leave Orphans which could affect upstream verification verifications 

 h3.    Actions: 

 h4. Upstream 

 *    Need an upstream verification that ensures there is no change in functionality with remove units and sync units 

 h4. Downstream 

 * Need to verify downstream verification verifications are not affected by this change/patch 
 * BZ verification will just be the removal of a large number of repos takes a short amount of time