Task #8297


Documentation relating to Directory Structure & File Systems

Added by wibbit about 3 years ago. Updated about 2 years ago.

Start date:
Due date:
% Done:


Estimated time:
Platform Release:
Sprint Candidate:


Ticket moved to GitHub: "pulp/pulpcore/1974":

Good morning all.

TLDR: Should the documentation be updated to reference DEPLOY_ROOT, and recommend that it is on a single partition, further ammendments to MEDIA_ROOT, WORKING_DIRECTORY, CHUNKED_UPLOAD_DIR, STATIC_ROOT or FILE_UPLOAD_TEMP_DIR are limited/restrictred to where it's actually pertinent and detailed information as to why you might choose to make that change with associated risks and caveats supplied where appropriate for that deployment style.

And the rest...

Re-reading the installation instructions and settings for Pulp3, I think there is a hole in relation to recommended partition layout, and associated settings.

We appear to have 3 documented variables (suggesting, that these are things we want to consider setting)

MEDIA_ROOT '/var/lib/pulp/media' WORKING_DIRECTORY PosixPath('/var/lib/pulp/tmp') CHUNKED_UPLOAD_DIR 'upload'

Being the curious fellow that I am, looking through dynaconf I also find the following variables.

DEPLOY_ROOT PosixPath('/var/lib/pulp') STATIC_ROOT PosixPath('/var/lib/pulp/assets') FILE_UPLOAD_TEMP_DIR PosixPath('/var/lib/pulp/tmp')

This appears to indicate the following layout.

/var/lib/pulp /var/lib/pulp/assets /var/lib/pulp/media/ /var/lib/pulp/media/??? (artifacts? 1TB on my current test install) /var/lib/pulp/media/upload /var/lib/pulp/tmp

The below note recommends MEDIA_ROOT and WORKING_DIRECTORY share the same filesystem.

The only other "documentation" that I can find is

This suggests that there is an expectation that at a minimum DEPLOY_ROOT is a single partition, sized appropriately to store all of pulp's content.

I'm purprosefully ignoring the question of whether CHUNKED_UPLOAD, WORKING_DIRECTORY or FILE_UPLOAD_TEMP_DIR would not be better suited to /var/spool or /var/cache

Actions #1

Updated by ekohl about 3 years ago

For myself I started a reference in our Puppet module: Since Pulp 3.10 this is also the default layout. Perhaps that documentation should be copied upstream.

In I've argued that DEPLOY_ROOT should not be documented right now. That's because it's an internal variable and dynaconf doesn't allow you to override it today. If you do, MEDIA_ROOT will not be updated. Documenting it could give a false impression that it can be changed.

STATIC_ROOT is something that does not need to live on the same filesystem. It is essentially a cache of all assets shipped by Pulp and its plugins. That should be documented but generally not something users change.

FILE_UPLOAD_TEMP_DIR is a built in Django setting and due to how Pulp works, it is indeed recommended to have on the same filesystem as MEDIA_ROOT. See for more information. In #7178 I implemented a check that should verify the directories are on the same filesystem.

With that check, running pulpcore-manager check --deploy should warn you. The problem is that there's a lot of warnings. In I started to analyze the ones I saw in our module. IMHO these should be silenced if they're irrelevant or solved if they are relevant. All of that should be in the default config rather than in some installer. Once that's done, running checks should become something any installer does and also a step in troubleshooting issues.

As a final note, if you do use SELinux then you should also take care of the right labels. By default takes care of this.

Actions #2

Updated by fao89 about 3 years ago

  • Tracker changed from Issue to Task
  • % Done set to 0
  • Severity deleted (2. Medium)
  • Triaged deleted (No)
  • Quarter set to Q1-2021
Actions #3

Updated by daviddavis almost 3 years ago

  • Quarter changed from Q1-2021 to Q2-2021
Actions #4

Updated by mdellweg almost 3 years ago

There's a list at the end of this docs page that might be needed updating here:

Actions #5

Updated by over 2 years ago

  • Quarter changed from Q2-2021 to Q3-2021
Actions #6

Updated by over 2 years ago

  • Quarter changed from Q3-2021 to Q4-2021
Actions #7

Updated by pulpbot about 2 years ago

  • Description updated (diff)
  • Status changed from NEW to CLOSED - DUPLICATE

Also available in: Atom PDF