Pulp

Thank you for filing this and for the PR. We definitely want to allow the Unix Domain Socket for the API and Content app. Is the actual limitation that the pulp_content_host and pulp_content_port are split?

Regarding the goal of de-duplicating the vars, the reason there is this duplication is around a theoretical use case that we don't fully achieve today. I want to tell it here to try to find a path towards it while also de-duplicating where possible.

use case: As a pulp installer user, I can use multiple hosts for the pulpcore-content gunicorn processes and multiple hosts for the pulpcore-api gunicorn processes. In large installations one host won't be enough. The idea with these being separate was that someone could configure it like this:

pulp_content_host: <content gunicorn load balancer dns/ip>
pulp_content_port: <content gunicorn load balancer port>
pulp_api_host: <api gunicorn load balancer dns/ip>
pulp_api_port: <api gunicorn load balancer port>

Then each host could bind to accept non-local hosts with:

pulp_content_bind: 0.0.0.0:24816
pulp_api_bind: 0.0.0.0:24817

The theoretical part is that there would need to be a load balancer manually deployed. One other option is to have all of the backend hosts configured as a group in nginx and apache and let the reverse proxy handle the load balancing. Neither one fully works today though.

Given these goals, and the goal of using unix socket domains, and de-duplication, what do you think we should do?

Is the actual limitation that the pulp_content_host and pulp_content_port are split?

Well, more the way we use it in the Nginx configuration, preventing the use of actual socket. Link: https://github.com/pulp/pulp_installer/blob/master/roles/pulp_webserver/templates/nginx.conf.j2#L26

Please let me know if my understanding is wrong, but this still can be accomplished without having the user to repeat himself. We can find thosse value out from the pulp_SERVICE_bind value. Using a snippet like this:

---
- hosts: localhost
  gather_facts: false
  tasks:
    - name: Extrapolate host and port when not UDS from pulp_service_bind
      set_fact:
        pulp_service_host: '{{ pulp_service_bind.split(":")[0] }}'
        pulp_service_port: '{{ pulp_service_bind.split(":")[1] }}'
      when: not pulp_service_bind.startswith('unix:')

    - name: Display values
      debug:
        msg: 'Pulp Service Bind Host: {{ pulp_service_host }} - Pulp Service Bindd Port: {{ pulp_service_port }}'
      when: not pulp_service_bind.startswith('unix:')`

spredzy wrote:

Is the actual limitation that the pulp_content_host and pulp_content_port are split?

Well, more the way we use it in the Nginx configuration, preventing the use of actual socket. Link: https://github.com/pulp/pulp_installer/blob/master/roles/pulp_webserver/templates/nginx.conf.j2#L26

I agree the thing that is blocking you is that the nginx config combines two variables. Consolidating those into one I think makes sense. Do you agree that would solve your immediate problem?

Please let me know if my understanding is wrong, but this still can be accomplished without having the user to repeat himself. We can find thosse value out from the pulp_SERVICE_bind value.

I agree it can be done but my concern is that if the nginx config and the bind addr are deduplicated we are locking into a single-host design for pulpcore-content and pulpcore-api (the gunicorn processes). This is because one bind address could only contain one host's info and nginx would have to use that. Does my concern make sense? Maybe we should de-duplicate these for now and tackle the multi-host problem as a separate piece of work, I'm not sure.

I want to find a way to unblock your work though without much delay. If you have a suggestion on a minimal set of changes that may be the most expedient way to resolve your issue. Either way, thank you for the great collaboration!

bmbouter wrote:

spredzy wrote:

Is the actual limitation that the pulp_content_host and pulp_content_port are split?

Well, more the way we use it in the Nginx configuration, preventing the use of actual socket. Link: https://github.com/pulp/pulp_installer/blob/master/roles/pulp_webserver/templates/nginx.conf.j2#L26

I agree the thing that is blocking you is that the nginx config combines two variables. Consolidating those into one I think makes sense. Do you agree that would solve your immediate problem?

Please let me know if my understanding is wrong, but this still can be accomplished without having the user to repeat himself. We can find thosse value out from the pulp_SERVICE_bind value.

I agree it can be done but my concern is that if the nginx config and the bind addr are deduplicated we are locking into a single-host design for pulpcore-content and pulpcore-api (the gunicorn processes). This is because one bind address could only contain one host's info and nginx would have to use that. Does my concern make sense? Maybe we should de-duplicate these for now and tackle the multi-host problem as a separate piece of work, I'm not sure.

I want to find a way to unblock your work though without much delay. If you have a suggestion on a minimal set of changes that may be the most expedient way to resolve your issue. Either way, thank you for the great collaboration!

Might I suggest a slightly different approach? One that meets both needs.

We have all 6 variables: pulp_content_bind pulp_api_bind pulp_content_host: pulp_content_port: pulp_api_host: pulp_api_port:

If pulp_content_bind and/or pulp_api_bind starts with "unix:": The individual host & ports values are ignored. pulp_webserver uses the values of pulp_content_bind and/or pulp_api_bind.

If pulp_content_bind and pulp_api_bind do not start with "unix:": The individual host & ports values are used by pulp_webserver, like they are now.

The implementation would presumably use internal variables to implement this into the template.

We could also just have the user specify "IP" or "UDS" in a variable like pulp_webserver_connection_mode, and then use the appropriate variables for that use mode (mode of pulp_webserver). "IP" uses certain vars, and "UDS" uses other vars.

Applied in changeset ansible-pulp3|afbf8264236a8673457c35051f1cb7f703b24fea.