Pulp

Story #7791

Updated by pulpbot almost 3 years ago

 

 **Ticket moved to GitHub**: "pulp/pulpcore/1941":https://github.com/pulp/pulpcore/issues/1941 




 ---- 


 If an artifact has gone missing or corrupted(bit rot) there is no way to re-upload it back to the filesystem 
 1. upload an artifact 
 2. rm /var/lib/pulp/artifacts/<some_artifact> or corrupt it 
 3. upload same artifact 
 4. 400 error with { 
     "non_field_errors": [ 
         "sha384 checksum must be unique." 
     ] 
 } 

 While we have /repair/ endpoint it will not work for the operations where artifact has no    remoteartifact. 

 ## Problem statements: 
 ### 1. If a file is missing it is impossible to upload a new one 
    * when saving artifact add try/except, look for existing one 
    * verify whether storage_path is an existing location if not update it with the newly uploaded bits 
    * Issue 400 due to duplicated artifact, but in addition **return the href** of the existing artifact. 
 ### 2. If a file is corrupted it is impossible to re-upload and replace it with a valid one 
 * **option1** Running repair can find corrupted files. It should remove the corrupted file to get back to the case outlined in 1. . 
      * Repair can be run against specific repo version, potentially can extend the functionality to repair a specific artifact/content 
 * **option2**    We could recalculate the checksum on all upload attempts. 
      * Might be a lot of overhead for a rare failure 
 * **option3** Introduce a flag which will be specified at upload time E.g. --repair, or --force, or --validate-checksum. 
 It will replace broken bits if checksum of the newly uploaded file matches a checksum in the DB. 
 The recalculation of a checksum will happen on_demand this way and not for every upload attempt. 

Back