File Support - Issue #3770: Pulp 3 is about 2x slower than pulp 2 in syncing a large file repo
RemositoryVersion's add_content and remove_content does not perform bulk operations
A cprofile report shows that a lot of time is being spent in RepositoryVersion.add_content() which interacts with the database on a unit-by-unit level. This is taking a long time. We need to improve the interface to perform add_content and remove_content via bulk operations.
1. Create a test in python that adds/removes X number of content units to a repo version
2. Benchmark the test
3. Update add_content and remove_content to support lists of content
4. Benchmark the change
I'll probably use different values of X starting with 1,000 and increasing up by factors of 10.
#3 Updated by daviddavis over 2 years ago
remove_content() to accept querysets. Here are the initial results for 1000 content units to a repo version:
add_content currently: 43.9s
add_content with bulk_create: 4.3s
remove_content currently: 44.4s
remove_content with a queryset: 0.5s
Please register to edit this issue