Project

Profile

Help

Story #3707

As a user, I can run multiple resource_managers for high availability

Added by bmbouter over 1 year ago. Updated about 1 month ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Platform Release:
Blocks Release:
Backwards Incompatible:
No
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

With the switch to RQ, the resource_manager became not-highly available. If you run two of them, bad things will happen.

We should make the resource_manager highly available again. there are two implementation options:

1. Use TaskLock again

Before the RQ transition we were using TaskLock: https://github.com/pulp/pulp/blob/968766e975c2eca00169470b8a028ec28c2a9274/pulpcore/pulpcore/app/models/task.py#L250-L280

This would be a PostgreSQL based table that only serves to elect a single master by one person writing the unique lock record.

2. Built something Redis based

This would be more something to contribute upstream to RQ.

3. Find something that we can wrap around the resource manager to provide a singleton function.


Checklist

History

#1 Updated by bmbouter over 1 year ago

  • Sprint Candidate changed from No to Yes

This will fix our high availability narrative so Pulp will have 0 single points of failure. It's a small amount of work for a big benefit. As such, I'm nominating for the next sprint.

#2 Updated by bmbouter over 1 year ago

  • Sprint Candidate changed from Yes to No

After talking with @dalley, we're going to do this work later relative to other higher priority work. I'm removing the sprint candidate flag.

#3 Updated by bmbouter over 1 year ago

  • Description updated (diff)

Updating to reflect that TaskLock is being removed from the codebase.

#4 Updated by bmbouter 6 months ago

  • Tags deleted (Pulp 3)

#5 Updated by bmbouter 6 months ago

  • Description updated (diff)
  • Sprint/Milestone set to 3.0

This was expressed as desirable for Katello's usage for the project itself (not Katello end users). So it's not a P1-P4, but it is important for them to adopt Pulp3 (and probably many others).

#6 Updated by bmbouter about 1 month ago

  • Sprint/Milestone deleted (3.0)

Please register to edit this issue

Also available in: Atom PDF