Project

Profile

Help

Story #4494

As a plugin writer, I can add custom installation steps

Added by ttereshc 9 months ago. Updated 7 months ago.

Status:
MODIFIED
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Platform Release:
Blocks Release:
Backwards Incompatible:
No
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 3 installer
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 53

Description

If any plugin has special installation steps, currently there is no way to add those per plugin basis.
E.g. for RPM plugin some build dependencies should be installed before createrepo_c gets pip-installed.


Related issues

Related to RPM Support - Story #4163: As a user, I can install pulp_rpm using the Ansible installer MODIFIED Actions
Blocked by Pulp - Story #4660: As an ansible-pulp user, I can configure my venv to have system-site-packages enabled MODIFIED Actions

History

#1 Updated by amacdona@redhat.com 9 months ago

What QE is doing with pre_tasks is one way to accomplish this. It's not ideal because it needs to be configured by the user, but I don't see a better way.

Another option could be to add to the plugin variables, like this:

pulp_install_plugins:
pulp_rpm:
app_label: rpm
required_packages:
- gcc
- make

This approach is a little cleaner, but still forces each user to know the dependencies of the pulp_rpm plugin. Note that with either approach, these values may vary depending on the package manager being used by the managed node OS.

#2 Updated by kersom 9 months ago

We could have variables to chose what plugin not to install, like create `defaults` pulp_rpm_install: yes, then this could be overwritten doing the playbook call.

ansible-playbook -vvv -i 192.168.122.138 -u root source-install-plugins.yml -e pulp_rpm_install no

This will give more control over what it will be installed.

Not sure if this kind of solution was discussed before.

#3 Updated by jsherril@redhat.com 9 months ago

My 2 cents:

  • The user shouldn't have to know about these things (or even that an additional playbook is required) The user needing to know anything other than 'i want plugin X' is asking too much.
  • Few plugins will require this (maybe i'm wrong?)
  • Its okay if there is plugin specific stuff in this module (structure it to make this easy to contribute to)

#4 Updated by ehelms@redhat.com 9 months ago

From my experiences, I'll offer up two approaches for plugin writers:

1) Treat ansible-pulp3 as the canonical source for installation and have plugin writers contribute storing all plugin enablement and install in the main pulp3 role
2) Create a role for every plugin and allow simple composition. Plugin writers can contribute their role to the repository itself or publish it independently to be included.

Flexibility can lead to complexity so I caution erring on the side providing a well-defined method more so than a flexible method for plugin authors at the outset.

#5 Updated by ttereshc 9 months ago

  • Related to Story #4163: As a user, I can install pulp_rpm using the Ansible installer added

#6 Updated by dkliban@redhat.com 9 months ago

  • Groomed changed from No to Yes
  • Sprint Candidate changed from No to Yes

#7 Updated by dkliban@redhat.com 9 months ago

  • Sprint set to Sprint 50

#8 Updated by bmbouter 9 months ago

The issue I see with custom plugin steps being added to ansible-pulp3 itself is that we won't be able to handle version changes since ansible-pulp3 can't be versioned with plugin.

Say in plugin foo 0.1 you have to run 'thiscommand' but with 0.2 you have to run 'thatcommand'. The current version of the installer have both so someone is going to end up broken.

For this reason, I believe having the plugin provide a custom role and playbook that calls ansible-pulp3 is the easiest and most viable solution.

#9 Updated by amacdona@redhat.com 9 months ago

@bmbouter, I think that makes sense. So a playbook would be:

- hosts: all
roles:
- pulp-rpm-presteps
- pulp-database
- pulp-workers
- pulp-resource-manager
- pulp-webserver
- pulp-content

Importantly, the pulp-rpm-presteps role would not be included in the ansible-pulp repository, and would be an entirely separate project.

Seems like this means there isn't really anything the installer needs to do other than suggest a role and/or pretasks as a documentation story.

#10 Updated by bmbouter 9 months ago

@asmacdo what you are describing sounds right.

#11 Updated by amacdona@redhat.com 9 months ago

This plan is fine for running ansible playbooks directly, but there is still a gap for pulplift, which doesn't allow the user to run a custom playbook before provisioning.

Maybe we could create a new directory of pretasks, and the user could configure which pretasks to include?

For example, as-is, a developer will still have to include custom pretasks (https://github.com/PulpQE/pulp-qe-tools/blob/master/pulp3/install_pulp3/source-install-plugins.yml#L26-L44) in a the playbook (https://github.com/pulp/pulplift/blob/master/playbooks/source-install.yml#L11), which is checked into git, so not ideal.

#12 Updated by bmbouter 9 months ago

@asmacdo, I see what you are saying.

What about having the rpm aspects install in pulplift using a post_install playbook? https://github.com/theforeman/forklift/#post-install-playbooks

#13 Updated by rchan 9 months ago

  • Sprint changed from Sprint 50 to Sprint 51

#14 Updated by ppicka 8 months ago

What about to keep role (or just playbook) with dependencies inside pulp_rpm repository (what I understood we want) and in installer (https://github.com/pulp/ansible-pulp/blob/master/roles/pulp/tasks/install.yml#L112 e.g. above this line) got something like

when: pulp_install_plugins.pulp_rpm is defined
and if true call the playbook from rpm repository. This possibly can avoid conflict in plugin versions and user from adding any pre-task or manually edit dependencies.

#15 Updated by bmbouter 8 months ago

@ppicka, +1 to what you are proposing here.

We should probably start with getting that extra 'rpm install' role into the pulp_rpm repo and then we can add that line here to call it during install time.

I think one of us should pick this up soon to work on it. What do you think?

#16 Updated by rochacbruno 8 months ago

I have done something similar few months ago using `include_tasks`

there are some related discussion on:

https://github.com/pulp/ansible-pulp/pull/51/files

#17 Updated by ppicka 8 months ago

  • Status changed from NEW to ASSIGNED

@bmbouter I assigned #4163 and will send PR for pulp_rpm repository first

but I see playbook will fit more as role is IMO so big for one task.

#18 Updated by bmbouter 8 months ago

@ppicka how are you going to have it install the dependencies for Pulp RPM?

After thinking about it I think we should have them install via the rpms available to Fedora and RHEL. This will limit RPM to only running on those distros but from what I'm hearing about libmodulemd not running on other distros I think it's unavoidable at this point.

@dalley does this sound right to you also?

#19 Updated by amacdona@redhat.com 8 months ago

  • Blocked by Story #4660: As an ansible-pulp user, I can configure my venv to have system-site-packages enabled added

#20 Updated by ipanova@redhat.com 8 months ago

  • Assignee set to ppicka

#21 Updated by rchan 8 months ago

  • Sprint changed from Sprint 51 to Sprint 52

#22 Updated by daviddavis 8 months ago

  • Sprint/Milestone set to 3.0

#23 Updated by bmbouter 8 months ago

  • Tags deleted (Pulp 3)

#24 Updated by rchan 7 months ago

  • Sprint changed from Sprint 52 to Sprint 53

#25 Updated by ppicka 7 months ago

  • Status changed from ASSIGNED to MODIFIED
  • % Done changed from 0 to 100

Applied in changeset commit:pulpcore-plugin|ad5c14604465cb18f2f17b1beb47d0cdb06ab988.

Please register to edit this issue

Also available in: Atom PDF