Issue #489
closed
When an task worker dies or restarts, work assigned to that worker is cancelled
Description
Description of problem:
When an RQ restarts (or dies) the tasks assigned to that worker get marked as cancelled. The cancelled state is correct, but users who dispatch work don't expect random cancelling of tasks so this isn't a great behavior.
To reproduce:
1. Start Pulp system with exactly 1 worker
2. Start a really long sync
3. List the tasks and observe that the task you submitted is in the running state
4. During the sync `kill -9` the worker
5. Start that worker up again
6. List the tasks again and observe that the task you submitted is in the cancelled state
Expected result: That the work you submitted to Pulp doesn't end up in the cancelled state when a worker dies. I expect it to be re-picked up by the worker and be in the running state.
Related issues
Updated by bmbouter over 9 years ago
- Subject changed from When a worker dies or restarts the tasks assigned to it are not processed to When a worker dies or restarts, work assigned to that worker is cancelled
- Description updated (diff)
- Status changed from ASSIGNED to NEW
- Assignee deleted (
bmbouter) - Priority changed from High to Low
- Severity changed from 3. High to 1. Low
- Version changed from 2.4.0 to 2.6.0
Updated by bmbouter almost 9 years ago
- Priority changed from Low to Normal
Raising priority as another Pulp user ran into a problem with this.
Also note, there is a new idea suggested by @asksol. https://github.com/celery/celery/issues/2492#issuecomment-177050069
We should try that.
Updated by bmbouter almost 8 years ago
- Related to Task #2527: Ensure Pulp 2.y works with both python-celery 3.1.z and 4.0.z added
Updated by jsherril@redhat.com almost 8 years ago
Katello is okay with this change. The result would be any tasks in katello that are assigned to a worker would no longer show as 'cancell'ed' forcing the user to resume them (or moving them to an errored state). I think this is better behavior and am okay with accepting potential differences in the future.
Updated by bmbouter over 5 years ago
- Status changed from NEW to CLOSED - WONTFIX
Updated by bmbouter over 5 years ago
Pulp 2 is approaching maintenance mode, and this Pulp 2 ticket is not being actively worked on. As such, it is being closed as WONTFIX. Pulp 2 is still accepting contributions though, so if you want to contribute a fix for this ticket, please reopen or comment on it. If you don't have permissions to reopen this ticket, or you want to discuss an issue, please reach out via the developer mailing list.
Updated by bmbouter over 5 years ago
- Subject changed from When a worker dies or restarts, work assigned to that worker is cancelled to When an RQ worker dies or restarts, work assigned to that worker is cancelled
- Description updated (diff)
- Status changed from CLOSED - WONTFIX to NEW
- Severity changed from 1. Low to 2. Medium
- Version deleted (
2.6.0) - Tags deleted (
Pulp 2)
Updated by dalley about 4 years ago
- Status changed from NEW to ASSIGNED
- Assignee set to dalley
Updated by dalley about 4 years ago
- Status changed from ASSIGNED to CLOSED - CURRENTRELEASE
Updated by bmbouter about 4 years ago
- Status changed from CLOSED - CURRENTRELEASE to NEW
Updated by dalley over 3 years ago
- Subject changed from When an RQ worker dies or restarts, work assigned to that worker is cancelled to When an task worker dies or restarts, work assigned to that worker is cancelled
- Assignee deleted (
dalley)
Added by ttereshc almost 3 years ago
Updated by ttereshc almost 3 years ago
- Status changed from NEW to MODIFIED
Applied in changeset pulp-2to3-migration|d00d7d67a055876130027908378c0952fc2adcf2.
.
Added forgotten changelog entry for 0.11.8
For PR https://github.com/pulp/pulp-2to3-migration/pull/482
closes #489