Project

Profile

Help

Story #3716

As a production Pulp user, I have an Ansible installer

Added by bmbouter over 1 year ago. Updated 8 months ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

31%

Platform Release:
Blocks Release:
Backwards Incompatible:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 3 installer
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

Motivation

1. Installation of Pulp is complicated and could be made much easier with an Ansible installer.

2. Clustered based installs are also very complicated. Typically users want to install some mixture of Pulp services on different hosts. Doing this consistently is time consuming and error prone. Ansible could install in clusered environments very well.

Usage

The installer will live as a collection of roles published via the 'pulp' user on Ansible Galaxy. That will cause it to be available here Each role will install a pulp component, and using an Ansible playbook the user can select which roles will run on any given host or host group. Here are the roles we foresee:

  • pulp3 - install the pulp virtualenv and any plugins
  • webserver - depends on pulp3. Installs the webserver of choice. Install systemd unit file for webserver. Set to autostart.
  • resource manager - depends on pulp3. Installs resource manager systemd unit file. Set to autostart
  • workers - depends on pulp3. Installs workers systemd unit files. Set to autostart
  • Redis - hopefully an existing Role on Galaxy named in our playbooks as a dependency
  • Postgres - hopefully an existing Role on Galaxy named in our playbooks as a dependency

Features

  • Generic plugin installation as specified by Ansible variables. Any plugin that only requires a pip install command and the migrations run will work
  • Install and configure Redis on a specific host in the inventory.
  • Install and configure Postgres on a specific host in the inventory.
  • Installs the virtualenv to be used in a role.
  • Can optionally install to the system packages without a virtualenv
  • Installs systemd units for a webserver, workers, or resource manager on one or more host in the inventory
  • Allows a choice of Nginx or Apache (defaults to nginx)
  • Allows for custom settings.py file provided by the user
  • Provides sparse settings.py by default
  • Idempotent

Assumptions

Dependency

This assumes that the installing the plugin will bring all of it's necessary dependencies including pulpcore.plugin and pulpcore itself. Specifically plugins are recommended to depend on pulpcore.plugin. That would bring in dependencies as follows:

Using Ansible variables, the Ansible installer will pip install pulp_ansible for example which will cause the installation of these packages: pulp_ansible ---> pulpcore.plugin ----> pulpcore ----> pulpcore.common

Storage

In multi-host environments Pulp has a shared storage requirement. The Ansible playbook does not automatically set these up. It assumes that shared storage is already setup correctly.

Root Access

At a few points in the playbook, root is required so it's assumed that the user the playbook will access has root access. Specifically root is required for the following:

  • To install Redis, Postgres, and the webserver, you'll need root
  • By default, the virtualenv will be created at /usr/local/pulpenv which typically requires root to write into
  • In the case that virtualenv is unset, it will install Python code to the system packages which also will require root
  • To create a user to run Pulp as will require root

Subtasks

Task #3717: Delete the extra reposNEW

Actions
Story #3719: Create the webserver roleMODIFIED

Actions
Story #3727: As a role user, I can have Nginx installed and configured to serve PulpMODIFIED

Actions
Story #3728: As a role user, I can have Apache installed and configured to serve PulpMODIFIEDehelms@redhat.com

Actions
Story #3723: Create the pulp3 roleMODIFIED

Actions
Story #3720: As a role user, I specify an Ansible variable that specifies the data directory Pulp will use. Defaults to '/var/lib/pulp/'MODIFIED

Actions
Story #3721: As a role user, server.yml behaviors are saneNEW

Actions
Task #3730: Create a resource manager roleMODIFIED

Actions
Task #3731: Create the workers roleMODIFIED

Actions
Task #3732: Have playbooks use a Postgresql roleMODIFIED

Actions
Task #3733: Have playbooks use a Redis roleMODIFIED

Actions
Story #3971: As a pulp3 role user, I can install pulpcore from sourceMODIFIEDamacdona@redhat.com

Actions
Issue #3973: As a pulp3 role user, production installs of pulpcore are handled by dependency resolutionMODIFIEDamacdona@redhat.comActions
Story #3974: As a pulp3-webserver role user, I can serve the REST API and the Content App from different machinesCLOSED - WONTFIX

Actions
Issue #3975: pulp3-workers role should not install RQCLOSED - WONTFIXActions
Story #3976: As a role user, I can install and configure a multi-machine pulpCLOSED - WONTFIX

Actions
Task #3977: Publish ansible roles to ansible-galaxyCLOSED - WONTFIX

Actions

History

#1 Updated by bmbouter over 1 year ago

  • Description updated (diff)

#2 Updated by bmbouter over 1 year ago

  • Description updated (diff)

Revised feature set claim

#3 Updated by bmbouter over 1 year ago

  • Description updated (diff)

#4 Updated by bmbouter over 1 year ago

  • Description updated (diff)

#5 Updated by bmbouter over 1 year ago

  • Description updated (diff)

Added root access as an assumption

#6 Updated by bmbouter over 1 year ago

  • Description updated (diff)

#7 Updated by Ichimonji10 over 1 year ago

Quite a bit of development work effort has been put into solving this issue. From where we currently stand, I think the next steps for the installer are as follows:

  1. Polish the existing code.
    • Merge existing PRs.
    • Merge the roles-dev branch back into the master branch. This ensures that the state of development is more obvious to people who don't have context with how development currently works on the roles-dev branch.
    • Fix install locations! For example: place pulp's files in /usr/local/lib/pulp instead of /opt/pulp or /usr/lib/pulp.
    • Normalize variable names and values. For example:
      • Make all variables begin with "pulp_" (as opposed to e.g. "pulp3_")
      • Make sure all roles have the same variable names.
      • Make sure all roles have the same default values set. Use unit tests?
    • Update all READMEs.
    • Discover whether installer installs from PyPI or from source right now. If there are any discrepancies, settle on just one strategy.
    • Make sure the "pulp" user is a system user, not a regular user.
  1. Create a Jenkins job that installs Pulp 3 on RHEL 7.
    • This is important because we want to make our sponsor happy.
    • This is important because the container images can be different from bare-metal/VM OS installations.
  1. Make installer capable of installing from PyPI or from source.
  2. Create a role that can install an arbitrary plugin, or many arbitrary plugins.
  3. Release somehow.
    • Make a git repository available?
    • Publish to galaxy via a monolithic repository?
    • Publish to galaxy via independent repositories?

#8 Updated by bmbouter 8 months ago

  • Tags deleted (Pulp 3)

Please register to edit this issue

Also available in: Atom PDF