Pulp

Is the problem here just the leaf-node? If it's just the leaf-node, then we can 'fix' this just by insuring we directly chmod() the end-result to the perms we want/need/expect. If any/all internediate dirs need the permissions as well, then there is some more-interesting work that has to happen.

ipanova@redhat.com , ttereshc - thoughts?

How about we use the systemd unit files to set the umask to 002?

Note about setgid:

Those perms drwxr-sr-x are 2755, where the leading 2 is setgid.

The setgid is getting inherited from the parent dir, so pulp_installer did its job of setting it. setgid cannot be controlled by umask, so it will not complicate this approach.

mdepaulo@redhat.com wrote:

How about we use the systemd unit files to set the umask to 002?

Interesting idea. We're explicitly setting umask for pulp-resource-manager in Pulp2 now:

system/pulp_resource_manager.service:
ExecStart=/usr/bin/celery worker -A pulp.server.async.app -n resource_manager@%%h
-Q resource_manager -c 1 --events --umask 18 --pidfile=/var/run/pulp/resource_manager.pid

Note about setgid:

Those perms drwxr-sr-x are 2755, where the leading 2 is setgid.

The setgid is getting inherited from the parent dir, so pulp_installer did its job of setting it. setgid cannot be controlled by umask, so it will not complicate this approach.

Yeah, concur, ownership should be fine, it's just the default-directory-perms that need to be addressed.

mdepaulo@redhat.com wrote:

How about we use the systemd unit files to set the umask to 002?

The problem with setting it outside code is, that umask can be/is set:

• at the service level in /etc/systemd/system/.service, which can be overridden by
• at the user-level, in /etc/systemd/user.conf and users/, which can be overridden by
• the system-level shell/env files /etc/bashrc /etc/profile and /etc/cshrc

Setting it explicitly in code, of course, trumps all of the above.

I'm experimenting on a pulp2 box. So far, I have not figured out which umask-default controls this particular scenario, and it makes me sad. I do have a fix in code that works. Experimentation continues.

Commentary/discussion with jsherrill:

On Wed, Sep 30, 2020 at 9:04 AM Justin Sherrill jsherril@redhat.com wrote:

Wouldn't this first approach also require a script to fix existing files/directories?

The pulp3 installer does exactly tyhat at install-time :

https://github.com/pulp/pulp_installer/blob/master/roles/pulp_common/tasks/install.yml#L107-L133

The difference I think is that this first approach would only need the script run once prior to migration. While the 2nd approach would require it to be run before every migration. I could see this taking a long time on a filesystem with 100s of thousands of files, or millions even. More so on a high latency file system. Since subsequent migrations are aimed to be as 'quick' as possible, i think we have to go with the first option?

The 'what if this is a HYOOGE filesystem" argument is persuasive :)

(Addendum to #c18 - "first approach" is "explicitly set/reset umask prior to makedirs() calls", "second approach" is "run script after any pulp2-activity post-pulp3-install")

One final(?) complicating factor to be aware of: in python2, os.makedirs(dir, mode=) will try to set the permissions on all directories it creates on the way to the leaf. In python3, that is no longer true - intermediate directories'permissions are driven just by umask.

This means that a sequence like

import os
mask = os.umask(0)
os.makedirs("/tmp/foo/bar/blech", mode=0770)

results in each of foo, bar, and blech ending up with permissions 770.

The same code in python3 results in foo and bar being 777, and only blech being 770.

Especially in testing, it's important to remember the environment the code will be running under. Pulp2 runs under Python2 and can therefore 'assume' the "mode= is for perms for any/all created dirs" behavior, where Pulp3 cannot.

Hopefully this note will save someone else from the 10 minutes of confusion I just suffered....

ggainey wrote:

mdepaulo@redhat.com wrote:

How about we use the systemd unit files to set the umask to 002?

The problem with setting it outside code is, that umask can be/is set:

• at the service level in /etc/systemd/system/.service, which can be overridden by
• at the user-level, in /etc/systemd/user.conf and users/, which can be overridden by

No, this affects systemd user mode, not system mode, which Pulp runs in.

• the system-level shell/env files /etc/bashrc /etc/profile and /etc/cshrc

No, one of the designs behind systemd is to prevent these from bleeding in. And for umask, that's actually a different issue entirely than environment variables.

Setting it explicitly in code, of course, trumps all of the above.

You're right, I just thought I'd propose the systemd unit files because:

1. it would be quicker/easier to implement.
2. If there was ever a need to configure it at runtime/install-time, it would be easier.

I'm experimenting on a pulp2 box. So far, I have not figured out which umask-default controls this particular scenario, and it makes me sad. I do have a fix in code that works. Experimentation continues.

To clarify, this is a variant of the 1st approach.