Project

Profile

Help

Story #4708

closed

Improve performance of publishing files with iso_distributor via the way of fast-forward

Added by Zhiming over 5 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
2.20.0
Groomed:
Yes
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 53
Quarter:

Description

Improve performance of publishing with iso_distributor via the way of fast-forward

This function, which is used to publishing a repo with iso_distributor, takes around 35 minutes (there are ~88k units in the repository)

The old implementation would generate symlinks for all units in the repository then copy to target published directories. If there are huge mount of units in the repository, the publish would be pretty slow due to those file operations.

Actually, just a few files are added/changed before every publish. So if just generate symlinks for those incremental or changed files, the publish will get faster.

On our installation, for a repo with 71K, this reduced the runtime of this publish from ~23 minutes to ~30 seconds.


Related issues

Related to RPM Support - Issue #4857: Pulp 2 Nightly Regression in test_iso_crud.ISOUpdateTestCase.test_allCLOSED - DUPLICATEActions

Also available in: Atom PDF