Story #4708
closed
Improve performance of publishing files with iso_distributor via the way of fast-forward
0%
Description
Improve performance of publishing with iso_distributor via the way of fast-forward
This function, which is used to publishing a repo with iso_distributor, takes around 35 minutes (there are ~88k units in the repository)
The old implementation would generate symlinks for all units in the repository then copy to target published directories. If there are huge mount of units in the repository, the publish would be pretty slow due to those file operations.
Actually, just a few files are added/changed before every publish. So if just generate symlinks for those incremental or changed files, the publish will get faster.
On our installation, for a repo with 71K, this reduced the runtime of this publish from ~23 minutes to ~30 seconds.
Related issues
Updated by ipanova@redhat.com over 5 years ago
Added by Zhiming over 5 years ago
Updated by ipanova@redhat.com over 5 years ago
- Status changed from POST to MODIFIED
Updated by ttereshc over 5 years ago
- Tracker changed from Issue to Story
- Assignee set to Zhiming
- % Done set to 0
- Groomed changed from No to Yes
- Sprint set to Sprint 53
Updated by ttereshc over 5 years ago
- Related to Issue #4857: Pulp 2 Nightly Regression in test_iso_crud.ISOUpdateTestCase.test_all added
Updated by ttereshc over 5 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
.
Improve performance of publish with iso_distributor via fast-forward
Only symlinks for those incremental or changed files are generated and copied to target directories, then the publish gets faster.
ref #4708 https://pulp.plan.io/issues/4708
update unit test because os.makedirs is now called conditionally
add missing parameters