Issue #2237
closed
Published repositories are copied instead of linked
Description
The pulp ostree publisher uses libostree pull-local which by nature will hard link files instead of copying them. However, the AtomicDirectoryPublishStep.process_main() in pulp uses shutil.copytree() instead of at least attempting to move the working-directory to the master/<timestamp> location. The result is, the ostree plugin pull-local creates an ostree repository using hard links but the copy operation creates a copy. Given the size of ostree repositories, this uses a ton of disk.
THIS NEEDS TO BE FIXED IN PLATFORM.
Should be doing something like this:
try:
rename()
except OSError as e:
if e.errno == errno.EXDEV:
copytree()
else:
raise
Updated by jortel@redhat.com over 7 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to jortel@redhat.com
- Sprint/Milestone set to 26
- Triaged changed from No to Yes
Updated by jortel@redhat.com over 7 years ago
- Status changed from ASSIGNED to POST
Added by jortel@redhat.com over 7 years ago
Added by jortel@redhat.com over 7 years ago
Revision 8e8d58fa | View on GitHub
Move working-directory to master location when permitted. closes #2237
Updated by jortel@redhat.com over 7 years ago
new PR against 2.10-dev
https://github.com/pulp/pulp/pull/2747
Updated by Ichimonji10 over 7 years ago
Can I get an example of where the source and destination files are on disk? In other words, where are files being copied/linked from, and where are files being copied/linked to?
Updated by jortel@redhat.com over 7 years ago
The files are copied from the distributor working directory. This is defined in server.conf [server] working_directory. By default it is: /var/cache/pulp. They are copied to: /var/lib/pulp/published/.
Example:
/var/lib/pulp/published/ostree/master/f24/1473278293.67/f24
├── config
├── extensions
├── objects
│ ├── 00
│ │ ├── 005d79aea748656db25d662f0f68faeda2a86618748942cee977b180a3ec57.filez
│ │ ├── 00bf0eea957ae3dfd852f015bd205ec08444ad17cc8efeb7d1694c90176a0d.filez
│ │ ├── 0120ed9e4b6a73bab7560a868df0fc4410e03f324000496c5bd317e4c103b4.filez
│ │ ├── 08c6b2a62472b5b41006babb176cf4370d83735c08c43088bd39824080d946.dirtree
.....
Updated by jortel@redhat.com over 7 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulp:pulp|8e8d58faeea3e7f6a26b824bafdbeff6870cb22e.
Updated by semyers over 7 years ago
- Status changed from 5 to CLOSED - CURRENTRELEASE
.
Move working-directory to master location when permitted. closes #2237