Project

Profile

Help

Task #2440

closed

Make celery processes work with pulp.tasking

Added by mhrivnak about 8 years ago. Updated almost 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Sprint:
Sprint 11
Quarter:

Description

@tteresch has also been working on this a lot with me. If there could be two assignees, she'd be one!

Pulp 3's pulp.tasking package does not work as-is with celery worker processes. Some re-arrangement has to happen to ensure that no pulp/django code gets used until the celery app has been instantiated and done it's "fixup" of django. It is also not allowed for pulp

Additionally, the pulp.tasking package itself could have a more intuitive organization. We are taking this opportunity to separate it into three categories of code:

  • code related to individual tasks
  • code related to running and managing services like workers and celerybeat
  • the minimum code necessary to instantiate and configure the celery app

The new layout will be:

pulp.tasking will contain celery_app and celery_instance
pulp.tasking.services for code that relates to workers, beat, etc.
pulp.tasking.tasks for Task base classes and the reservation tasks
pulp.tasking.util for get_current_task_id() and cancel()
pulp.app.tasks for pulp tasks that do stuff with pulp data

Also available in: Atom PDF