Story #7005: As a user, I can run a single stably-named Ansible role to install Pulp - Pulp

Actions

Send by e-mail Copy link

Story #7005

closed

As a user, I can run a single stably-named Ansible role to install Pulp

Added by mdepaulo@redhat.com almost 4 years ago. Updated almost 4 years ago.

Status:

CLOSED - CURRENTRELEASE

Priority:

Normal

Assignee:

Category:

Installer - Moved to GitHub issues

Sprint/Milestone:

3.5.0

Start date:

Due date:

% Done:

100%

Estimated time:

Platform Release:

Groomed:

Sprint Candidate:

Tags:

Sprint:

Quarter:

Description

We should restructure Pulp Installer's Ansible roles:

pulp (common components and pulp-api. pulp-api is actually a boolean)
pulp_redis (depends: pulp)
pulp_database (depends: pulp) (Has option to install db via external role. Always runs migrations.)
pulp_content (undeclared dep: pulp)
pulp_resource_manager (depends: pulp_redis, undeclared dep: pulp)
pulp_workers (depends: pulp_redis, undeclared dep: pulp)
pulp_webserver (depends: pulp)
pulp_devel (depends: pulp) (optional)

Where users are supposed to run this role list:

    - pulp_database
    - pulp_workers
    - pulp_resource_manager
    - pulp_webserver
    - pulp_content
    - pulp_devel # (optional)

Into the following:

pulp_common (common components) (calls prereqs roles)
pulp_redis
pulp_database (Has option to install db via external role. Always runs migrations.)
pulp_api (depends: pulp_common)
pulp_content (depends: pulp_common)
pulp_resource_manager (depends: pulp_common)
pulp_workers (undeclared dep: pulp_common)
pulp_webserver (depends: pulp_common)
pulp_devel (depends: pulp_common) (optional)
pulp_complete: (depends: everything except pulp_devel)

So then users will just run:

* pulp_complete
* pulp_devel # (optional)

Over time, we can consider adding other meta roles with fewer components. Like one for just the pulp's own services themselves.

Users can still emulate the old behavior, but with longer runtime on dynamic include, by running a slightly different list:

    - pulp_redis** # (New)
    - pulp_database
    - pulp_workers
    - pulp_resource_manager
    - pulp_webserver
    - pulp_content
    - pulp_api # (New)
    - pulp_devel # (optional)

Also, if we ever modify the roles again, users can still just run pulp_complete.

The molecule CI will be updated so that the "static" scenarios will install "pulp_complete"m, while the "dynamic" scenarios will specify the several roles.

Related issues

Actions

Copy link

Updated by mdepaulo@redhat.com almost 4 years ago

Description updated (diff)

Actions

Copy link

Updated by mdepaulo@redhat.com almost 4 years ago

Tracker changed from Task to Story
Subject changed from Restructure pulp_installer roles to As a user, I can run a single stable-named Ansible role to install Pulp
Description updated (diff)

Actions

Copy link

Updated by mdepaulo@redhat.com almost 4 years ago

Description updated (diff)

Actions

Copy link

Updated by mdepaulo@redhat.com almost 4 years ago

Description updated (diff)

Actions

Copy link

Updated by mdepaulo@redhat.com almost 4 years ago

Blocks Story #7006: As a user, I can use a single systemd service to restart Pulp added

Actions

Copy link

Updated by mdepaulo@redhat.com almost 4 years ago

Description updated (diff)

Actions

Copy link

Updated by mdepaulo@redhat.com almost 4 years ago

I'm worried "pulp_complete" would be confused with "complete set of plugins".

Other possible names: pulp_meta pulp_all_in_one pulp_allinone pulp_server pulp_single_server

Actions

Copy link

Updated by mdepaulo@redhat.com almost 4 years ago

Description updated (diff)

Actions

Copy link

Updated by bmbouter almost 4 years ago

mdepaulo@redhat.com wrote:

I'm worried "pulp_complete" would be confused with "complete set of plugins".

Other possible names: pulp_meta pulp_all_in_one pulp_allinone pulp_server pulp_single_server

From this list I like pulp_single_server and pulp_server the most. I'll also throw pulp_all_services out there too. In the future we want to have the installer provide cluster installs, what name can we select here that will help contrast against that? In the future clustered install mode we would not use this name right? If so, then having it call out "single_server" or "all_services" may be the best. Wdyt?

Actions

Copy link

#10

Updated by mdepaulo@redhat.com almost 4 years ago

bmbouter wrote:

mdepaulo@redhat.com wrote:

I'm worried "pulp_complete" would be confused with "complete set of plugins".

Other possible names: pulp_meta pulp_all_in_one pulp_allinone pulp_server pulp_single_server

From this list I like pulp_single_server and pulp_server the most. I'll also throw pulp_all_services out there too. In the future we want to have the installer provide cluster installs, what name can we select here that will help contrast against that? In the future clustered install mode we would not use this name right? If so, then having it call out "single_server" or "all_services" may be the best. Wdyt?

I agree with "pulp_all_services" for those reasons. I'll start on this now.

Actions

Copy link

#11

Updated by spredzy almost 4 years ago

I love this story. Yes an hundred times.

I have several comments, feel free to incorporate the one you find relevant and discard the other.

On the role separation¶

I have a small nit, I would vote in splitting the pulp_database role in two pieces.

pulp_database (that is basically the content of install_db playbook) - And that people might feel free to not call at all.
pulp_database_configuration (that is the migration really taking place) - That is mandatory to be run.

Name of the meta role¶

I would go with pulp_allinone, because really this is what it is, you call this role and you get a fully functionnal pulp system (redis, db, webserver and pulp services) up and running.

To me, pulp_all_services would be something like:

pulp_content
pulp_api
pulp_resource_manager
pulp_workers
pulp_database_configuration

This way people can come with their pre existing services Database, Redis and Webserver and still have a fully functional pulp backend (cluster or not).

You might be wondering why I am putting webserver out of the list ? The way it is managed today pretty much impose an nginx conf with a single server. What if I am a customer with an already existing fleet of Nginx servers that are the "only" nginx my company allows us to use as proxy entrance point (we already have monitoring, ha and ops practices for those) can I just integrate it in them ?

Technically, on top of already having the role, we would simply need to provide the nginx.conf in the doc so people can integrate it (Affter modiffication) to their already running nginx and have them point at the proper backend.

I feel the two above point would make the installer gain in flexibility and also ease the way into incorporating intro an already existing IT system (where thing are already backed up, monitored and all) rather that having pulp using their own services for its needs.

Actions

Copy link

#12

Updated by mdepaulo@redhat.com almost 4 years ago

Subject changed from As a user, I can run a single stable-named Ansible role to install Pulp to As a user, I can run a single stably-named Ansible role to install Pulp

Actions

Copy link

#13

Updated by mdepaulo@redhat.com almost 4 years ago

Description updated (diff)

Actions

Copy link

#14

Updated by mdepaulo@redhat.com almost 4 years ago

Description updated (diff)

Actions

Copy link

#15

Updated by mdepaulo@redhat.com almost 4 years ago

spredzy wrote:

I love this story. Yes an hundred times.

I have several comments, feel free to incorporate the one you find relevant and discard the other.

On the role separation¶

I have a small nit, I would vote in splitting the pulp_database role in two pieces.

pulp_database (that is basically the content of install_db playbook) - And that people might feel free to not call at all.

pulp_database_configuration (that is the migration really taking place) - That is mandatory to be run.

I was thinking of doing something like this as well.

More on the why below

Name of the meta role¶

I would go with pulp_allinone, because really this is what it is, you call this role and you get a fully functionnal pulp system (redis, db, webserver and pulp services) up and running.

To me, pulp_all_services would be something like:

pulp_content

pulp_api

pulp_resource_manager

pulp_workers

pulp_database_configuration

This way people can come with their pre existing services Database, Redis and Webserver and still have a fully functional pulp backend (cluster or not).

I plan to introduce another meta role that includes that set as well.

I was leaning against pulp_allinone because it kind of implies all plugins.

Someone else pointed out that "pulp_all_services" makes sense because we refer to everything as a service, including the database. As an SOA.

I'm thinking of naming the meta role for those 5 "pulp_services".

You might be wondering why I am putting webserver out of the list ? The way it is managed today pretty much impose an nginx conf with a single server. What if I am a customer with an already existing fleet of Nginx servers that are the "only" nginx my company allows us to use as proxy entrance point (we already have monitoring, ha and ops practices for those) can I just integrate it in them ?

Very good point. I am aware of this. It is a big deal for the kubernetes operator (pulp-operator).

Technically, on top of already having the role, we would simply need to provide the nginx.conf in the doc so people can integrate it (Affter modiffication) to their already running nginx and have them point at the proper backend.

Yeah, this is a problem for the kubernetes operator. generic k8s ingresses don't support it, or even the prior set of features we needed before the snippets (probably). The solution (for now, when I resume work on it) is to have Pulp's own webserver, and the kubernetes ingresses in front of it.

I feel the two above point would make the installer gain in flexibility and also ease the way into incorporating intro an already existing IT system (where thing are already backed up, monitored and all) rather that having pulp using their own services for its needs.

Agreed, I'll do them in separate PRs/issues though (the outstanding PR is getting harder to merge everytime another PR is merged.)

Added by Mike DePaulo almost 4 years ago

Revision dd70e48b | View on GitHub

As a user, I can run a single stably-named Ansible role

to install Pulp: pulp_all_services

Implementation includes:

The new role pulp_all_services, does nothing but depend.
pulp role split into pulp_common and pulp_api.
Removal of var pulp_install_api_service (due to it now being controlled by the inclusion of the pulp_api service.)
A new list of roles for users to run if they wish to install Pulp, starting with pulp_redis & pulp_database, then the Pulp services.
All Pulp service roles & pulp_webserver (temporarily) declare their existing dependency on pulp_common.
Many incorrect docs updates were fixed.
Fix misc unrelated yamllint errors.

fixes: #7005

Added by Mike DePaulo almost 4 years ago

Revision dd70e48b | View on GitHub

As a user, I can run a single stably-named Ansible role

to install Pulp: pulp_all_services

Implementation includes:

The new role pulp_all_services, does nothing but depend.
pulp role split into pulp_common and pulp_api.
Removal of var pulp_install_api_service (due to it now being controlled by the inclusion of the pulp_api service.)
A new list of roles for users to run if they wish to install Pulp, starting with pulp_redis & pulp_database, then the Pulp services.
All Pulp service roles & pulp_webserver (temporarily) declare their existing dependency on pulp_common.
Many incorrect docs updates were fixed.
Fix misc unrelated yamllint errors.

fixes: #7005

Actions

Copy link

#16