Project

Profile

Help

Task #2479

Update systemd tasking service files to use new Pulp 3 locations

Added by semyers almost 3 years ago. Updated 6 months ago.

Status:
MODIFIED
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

100%

Platform Release:
Blocks Release:
Backwards Incompatible:
No
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 18

Description

Now that the tasking system is up and running, we'd like to get those services working automatically via systemd. In order to do that, we should take the existing systemd unit files and update them to use the new paths.

Here are the unit files related to the tasking system in pulp 2:
https://github.com/pulp/pulp/tree/3.0-dev/server/usr/lib/systemd/system

The reserved resource worker service files are created by the manage_workers script:
https://github.com/pulp/pulp/tree/3.0-dev/server/pulp/server/async/manage_workers.py

Here are the current updated app invocations in Pulp 3, which hopefully won't change before we start this task:

# celery beat
/usr/bin/celery beat -A pulp.tasking.celery_instance.celery -S pulp.tasking.services.scheduler.Scheduler

# resource manager worker
/usr/bin/celery worker -A pulp.tasking.celery_app -n resource_manager@<i> -Q resource_manager -c 1 --events --umask 18 --pidfile=/var/run/pulp/resource_manager.pid --heartbeat-interval=30

# reserved resource workers
/usr/bin/celery worker -n reserved_resource_worker-<i>@<hostname> -A pulp.tasking.celery_app:celery -c 1 --events --umask 18 --pidfile /var/run/pulp/reserved_resource_worker-<i>.pid --heartbeat-interval=30

Systemd provides an "instancing" mechanism that we could potentially use for the reserved resource workers, if we'd like get away from using the "manage_workers" script, documented here:
https://www.freedesktop.org/software/systemd/man/systemd.unit.html (look for getty.service@ and the meaning of the %i expansion in unit files)

Once these unit files are in place, the ansible dev playbook will need to be updated to install these unit files and start/stop these services using the aliases defined in its .bashrc file.


Checklist


Related issues

Blocked by Pulp - Task #2413: Fix dev environment setup for Pulp 3.0 MODIFIED Actions

Associated revisions

Revision 24714937 View on GitHub
Added by fdobrovo over 2 years ago

Update systemd tasking service files

closes #2479
https://pulp.plan.io/issues/2479

Revision 24714937 View on GitHub
Added by fdobrovo over 2 years ago

Update systemd tasking service files

closes #2479
https://pulp.plan.io/issues/2479

Revision 24714937 View on GitHub
Added by fdobrovo over 2 years ago

Update systemd tasking service files

closes #2479
https://pulp.plan.io/issues/2479

Revision 1f981a75 View on GitHub
Added by fdobrovo over 2 years ago

Ansible deploy pulp3 systemd services

re #2479
https://pulp.plan.io/issues/2479

History

#1 Updated by semyers almost 3 years ago

  • Description updated (diff)

Some of the work has already been done on 3.0-dev, so I think this task is mainly about getting the service files into the right place and making the dev playbook use them correctly.

#2 Updated by dkliban@redhat.com almost 3 years ago

  • Groomed changed from No to Yes
  • Sprint Candidate changed from No to Yes

#3 Updated by semyers almost 3 years ago

  • Blocked by Task #2413: Fix dev environment setup for Pulp 3.0 added

#4 Updated by mhrivnak over 2 years ago

  • Priority changed from Normal to High

#5 Updated by mhrivnak over 2 years ago

  • Sprint/Milestone set to 37

#6 Updated by fdobrovo over 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fdobrovo

#7 Updated by dalley over 2 years ago

I just want to note that the app invocations have, indeed, changed. Make sure to use the form of the ones in the current tree instead of the ones described in this issue. I believe the only difference is that the heartbeat interval is no longer specified.

re: "Here are the current updated app invocations in Pulp 3, which hopefully won't change before we start this task:"

#8 Updated by bmbouter over 2 years ago

I believe the --heartbeat-interval options should all be removed

#9 Updated by fdobrovo over 2 years ago

Yeah I think I have seen it removed from the current systemd files. Also one thing that came into my mind and I don't know how to solve it. It would be nice to specify requirements, such as when starting the worker ensure that the broker is started or resource manager is running. But I'm thinking in our dev environment we may want to not run such things.

#10 Updated by bmbouter over 2 years ago

We can't add those unit file dependencies because any installation could be a clustered installation and we wouldn't want a broker started on each machine for example.

#12 Updated by bmbouter over 2 years ago

With a PR available this should be at POST unless there are other PRs that still need to be made.

#13 Updated by fdobrovo over 2 years ago

It's waiting for pulp/devel PR.

#14 Updated by fdobrovo over 2 years ago

  • Status changed from ASSIGNED to POST

#15 Updated by fdobrovo over 2 years ago

As dalley pointed out pulp-dev.py already installs the systemd files is it used during vagrant up?

#16 Updated by fdobrovo over 2 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#17 Updated by bmbouter over 1 year ago

  • Sprint set to Sprint 18

#18 Updated by bmbouter over 1 year ago

  • Sprint/Milestone deleted (37)

#19 Updated by daviddavis 6 months ago

  • Sprint/Milestone set to 3.0

#20 Updated by bmbouter 6 months ago

  • Tags deleted (Pulp 3)

Please register to edit this issue

Also available in: Atom PDF