Issue #4602
closed/var/lib/pulp/tmp/ seems to violate the FHS
Description
Pulp 3 currently uses /var/lib/pulp/tmp/ . This seems to be configurable as WORKING_DIRECTORY .
However, I re-reviewed the FHS (Filesystem hierarchy standard, which is part of the Linux Standard Base & approximately followed by flavors of unix). It should probably not be that directory. It should probably be /var/tmp/pulp/ .
https://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#varlibVariableStateInformation
https://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#vartmpTemporaryFilesPreservedBetwee
A good question to ask "can a system administrator, at their discretion, delete the contents of WORKING_DIRECTORY whlle Pulp is stopped without breaking/resetting their Pulp application state?" If "yes", then it should be moved to /var/tmp/pulp/ .
Note that Pulp previously stopped using /var/cache/pulp (a different PATH, but FHS compliant) to consolidate on /var/lib/pulp/tmp in issue 3406.
https://pulp.plan.io/issues/3406
However, this task is about the entire usage of the directory.
Updated by bmbouter over 5 years ago
One consideration to help inform a decision in this area. Subdirectories inside of the temporary directory is where downloaders running inside tasks write their data to. In almost all cases after saving the file there it is then moved in to place. Typically this place is on the local filesystem in the /var/lib/pulp/artifacts/... area. One performance consideration is that the copying of the file to the second location should not require the bits to be written out again. That is desirable at least.
One other FYI is that the final location can't be known until after the data's sha256 hash is calculated which produces the filepath where it is to be stored at. So we can't fully trust the final location until we already have saved all of the data to disk in the tmp area.
Just some things to think about along with this change.
Updated by dkliban@redhat.com over 5 years ago
When using S3, the temporary location doesn't matter. However, when using local storage, you would definitely get better performance if the temp directory was on the same physical device.
Updated by dkliban@redhat.com over 5 years ago
What if the default location for the temp directory was /var/tmp/pulp? At the same time the installer could create a symlink from the rest of pulp storage to there to improve performance.
Updated by bmbouter over 5 years ago
The great thing about keeping them both inside /var/lib/pulp/ is that the installer doesn't need the extra responsibility so it will provide the performance benefit natively.
The best thing I can think of to do is to document this in the docs. Perhaps with a ..note sphinx option that describes that hosting the working directory on the same volume as your backend storage when hosting artifacts locally on that storage?
Isn't it that any python code that wants to use temp directories will still get /var/tmp/ right? So in that way Pulp is using the Django temp directory feature to save staged data there and not really as the FHS temp dir. Is this right?
Updated by amacdona@redhat.com over 5 years ago
- Triaged changed from No to Yes
- Sprint set to Sprint 51
Updated by bmbouter over 5 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to bmbouter
- Tags Pulp 3 added
Added by bmbouter over 5 years ago
Updated by bmbouter over 5 years ago
- Status changed from ASSIGNED to POST
PR available at: https://github.com/pulp/pulpcore/pull/86
Updated by bmbouter over 5 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulpcore|eee07e3608026c6f7ddee1270ede4a599e6bf52b.
Updated by bmbouter over 5 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulpcore|01d5739d66fe742802030e3be834e761a9248aa9.
Updated by bmbouter over 4 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Deployment note on volumes for working dir
There was not enough info for users to understand the relationship between the WORKING_DIRECTORY and MEDIA_ROOT settings. This note clarifies this.
https://pulp.plan.io/issues/4602 closes #4602