Issue #8637
closed
Issue #8912: [EPIC] Issues with the traditional tasking system
Possible race condition where task's reservations get deleted
Description
This was something I noticed while working on #8603. I haven't actually reproduced it so it's purely theoretical.
The TaskReservedResource has a cascade delete on its ReservedResource FK, which could cause a race condition:
- Task 1 confirms that the reserved resource has no other tasks
- Task 2 reserves the resource
- Task 1 calls delete which cascades and deletes Task 2's reservations of the resource
Related issues
Updated by daviddavis over 3 years ago
- Related to Issue #8603: possible tasking race condition: update or delete on table "core_reservedresource" violates foreign key constraint "core_taskreservedres_resource_id_ee0b7c62_fk_core_rese" on table "core_taskreservedresource" added
Updated by dalley over 3 years ago
- Related to Issue #8568: Overall task status remains in 'running' state while all the reports are in the final state added
Updated by fao89 over 3 years ago
- Triaged changed from No to Yes
- Sprint set to Sprint 95
Updated by mdellweg over 3 years ago
Is the cascaded delete needed or even useful here? Isn't the cleanup process, that the association (TaskReservedResource) is deleted first and the then the ReservedResource is cleaned up if no longer needed? If we change the cascade to protect here, it should be fine, right? And if another task is assigned to the Resource in the meantime, we get a ConsistencyException we can simply ignore.
Updated by bmbouter over 3 years ago
I want to confirm the thinking here. When you say "the cleanup process, that the association (TaskReservedResource) is deleted first and the then the ReservedResource is cleaned up " you mean here right?.
If that's the case then I agree the cascade delete isn't needed. So say we got rid of the cascade delete ... couldn't this race condition still happen just between these two lines?.
Updated by daviddavis over 3 years ago
bmbouter wrote:
I want to confirm the thinking here. When you say "the cleanup process, that the association (TaskReservedResource) is deleted first and the then the ReservedResource is cleaned up " you mean here right?.
If that's the case then I agree the cascade delete isn't needed. So say we got rid of the cascade delete ... couldn't this race condition still happen just between these two lines?.
If you go back to my example, then when task 1 goes to clean up the ReservedResource, then a cascade protect from task 2 will prevent the ReservedResource from being deleted. This would avoid the race condition.
Updated by bmbouter over 3 years ago
Ah this does make sense @david. +1 to this plan, let's change this line to PROTECT.
Updated by daviddavis over 3 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to daviddavis
Updated by pulpbot over 3 years ago
- Status changed from ASSIGNED to POST
Added by daviddavis over 3 years ago
Updated by daviddavis over 3 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulpcore|cf2cde7e583c987154aa4bf90c124c32898c1151.
Updated by pulpbot over 3 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
.
Fix race condition during reserved resource cleanup
fixes #8637