Issue #489
Updated by bmbouter over 5 years ago
Description of problem: When an RQ a worker restarts (or dies) the tasks assigned to that worker get marked as cancelled. The cancelled state is correct, but users who dispatch work don't expect random cancelling of tasks so this isn't a great behavior. This affects a minimal number of tasks so it's not a big deal, but fixing it should happen in Celery [0] not Pulp. To reproduce: 1. Start Pulp system with exactly 1 worker 2. Start a really long sync 3. List the tasks and observe that the task you submitted is in the running state 4. During the sync `kill -9` the worker 5. Start that worker up again 6. List the tasks again and observe that the task you submitted is in the cancelled state Expected result: That the work you submitted to Pulp doesn't end up in the cancelled state when a worker dies. I expect it to be re-picked up by the worker and be in the running state. The root cause is because the CELERY_WORKER_DIRECT feature doesn't use durable queues. An upstream bug [0] has been filed, and this bug tracks the inclusion of an upstream fix into Pulp. [0]: https://github.com/celery/celery/issues/2492
.