Issue #8637
closedIssue #8912: [EPIC] Issues with the traditional tasking system
Possible race condition where task's reservations get deleted
Description
This was something I noticed while working on #8603. I haven't actually reproduced it so it's purely theoretical.
The TaskReservedResource
has a cascade delete on its ReservedResource
FK, which could cause a race condition:
- Task 1 confirms that the reserved resource has no other tasks
- Task 2 reserves the resource
- Task 1 calls delete which cascades and deletes Task 2's reservations of the resource
Related issues
Updated by daviddavis over 3 years ago
- Related to Issue #8603: possible tasking race condition: update or delete on table "core_reservedresource" violates foreign key constraint "core_taskreservedres_resource_id_ee0b7c62_fk_core_rese" on table "core_taskreservedresource" added
Updated by dalley over 3 years ago
- Related to Issue #8568: Overall task status remains in 'running' state while all the reports are in the final state added
Updated by fao89 over 3 years ago
- Triaged changed from No to Yes
- Sprint set to Sprint 95
Updated by mdellweg about 3 years ago
Is the cascaded delete needed or even useful here? Isn't the cleanup process, that the association (TaskReservedResource) is deleted first and the then the ReservedResource is cleaned up if no longer needed? If we change the cascade to protect here, it should be fine, right? And if another task is assigned to the Resource in the meantime, we get a ConsistencyException we can simply ignore.
Updated by bmbouter about 3 years ago
I want to confirm the thinking here. When you say "the cleanup process, that the association (TaskReservedResource) is deleted first and the then the ReservedResource is cleaned up " you mean here right?.
If that's the case then I agree the cascade delete isn't needed. So say we got rid of the cascade delete ... couldn't this race condition still happen just between these two lines?.
Updated by daviddavis about 3 years ago
bmbouter wrote:
I want to confirm the thinking here. When you say "the cleanup process, that the association (TaskReservedResource) is deleted first and the then the ReservedResource is cleaned up " you mean here right?.
If that's the case then I agree the cascade delete isn't needed. So say we got rid of the cascade delete ... couldn't this race condition still happen just between these two lines?.
If you go back to my example, then when task 1 goes to clean up the ReservedResource, then a cascade protect from task 2 will prevent the ReservedResource from being deleted. This would avoid the race condition.
Updated by bmbouter about 3 years ago
Ah this does make sense @david. +1 to this plan, let's change this line to PROTECT.
Updated by daviddavis about 3 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to daviddavis
Updated by pulpbot about 3 years ago
- Status changed from ASSIGNED to POST
Added by daviddavis about 3 years ago
Updated by daviddavis about 3 years ago
- Status changed from POST to MODIFIED
Applied in changeset pulpcore|cf2cde7e583c987154aa4bf90c124c32898c1151.
Updated by pulpbot about 3 years ago
- Status changed from MODIFIED to CLOSED - CURRENTRELEASE
Fix race condition during reserved resource cleanup
fixes #8637