Project

Profile

Help

Issue #907

closed

Resource reservations are not released properly

Added by bmbouter about 7 years ago. Updated about 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
1. Low
Version:
2.5
Platform Release:
2.5.4
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Quarter:

Description

The pulp tasking system of 2.5.x has a defect where resource reservations are not cleaned up. I have verified this does not affect 2.6.0 or newer versions of Pulp.

This causes workers to be considered reserved when they are not. This doesn't cause a correctness problem, but could potentially causes work to be incorrectly routed to workers who are already busy with other work. The workaround is to restart the Pulp worker with the vestige reservation which causes that worker to release all associated resource reservations.

To reproduce we need to look at the internal state of pulp as follows:

0) Install any 2.5.x version of Pulp.
1) Start all Pulp services.
2) Create a repo and sync it like this:

pulp-admin -u admin -p admin rpm repo create --repo-id zoo --relative-url zoo --feed http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
pulp-admin -u admin -p admin rpm repo sync run --repo-id zoo

3) After the task is fully finished the reserved_resource collection in the pulp database in MongoDB should be empty, but instead you'll find a record like this one:

{ "_id" : "repository:zoo" , "_ns" : "reserved_resources" , "num_reservations" : 1 , "assigned_queue" : "reserved_resource_worker-0@example.com.dq"}

4) If you run the sync again you'll see the num_reservations increase 1 for each time you run the sync.

Expected results: I that when work that requires a reservation completes and there is no other work in the system that the reserved_resource collection will be empty.

Actions #1

Updated by bmbouter about 7 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to bmbouter

Added by bmbouter about 7 years ago

Revision 1964dbde

Resource reservations now release properly

https://pulp.plan.io/issues/907

closes #907

Added by bmbouter about 7 years ago

Revision 1964dbde

Resource reservations now release properly

https://pulp.plan.io/issues/907

closes #907

Actions #2

Updated by bmbouter about 7 years ago

  • Status changed from ASSIGNED to MODIFIED
  • % Done changed from 0 to 100
Actions #3

Updated by bmbouter about 7 years ago

  • Description updated (diff)
Actions #4

Updated by cduryee about 7 years ago

  • Triaged changed from No to Yes

issue is already in modified state, setting triage to true.

Actions #5

Updated by bmbouter over 6 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Actions #6

Updated by bmbouter about 3 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF