Issue #9243
closed
30-50% re-sync performance regression due to touch() of content and artifacts during sync
Description
"touching" existing content and artifacts individually has led to a slowdown on re-syncs of between 30-50% depending on the size of the repo, due to performing (potentially) tens of thousands of extra queries. Copy is also likely impacted severely however I did not test it.
Bulk touch functionality was added to pulpcore by [0], but we must take advantage of it to eliminate the slowdown.
See the following spots where touch() is called in a loop:
Additionally it would be good to eliminate the extra query here: https://github.com/pulp/pulpcore/blob/feede7bb08b1e3107766ce534433cb7867fe52bd/pulpcore/content/handler.py#L715
As this was a significant performance regression introduced by 3.14, we should consider backporting the fixes. This will include [0] as well which will require a small portion of the refactor introduced here [1]
Related issues
.
Use bulk touch() when processing Artifacts and Content
This patch only addresses the inefficiencies of the Stages API. Another patch is needed to address the inefficiency of the repository version modify operation.
closes: #9243