Project

Profile

Help

Issue #2770

Tasks stuck in waiting state if received while qpidd is down

Added by bmbouter over 4 years ago. Updated almost 3 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
2.13.2
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 20
Quarter:

Description

To reproduce:

1. Start a normal, healthy Pulp system.
2. sync the zoo repo to make sure your system is health. Ensure this completes
3. Observe that there are 0 currently running tasks using pulp-admin tasks list
4. Stop qpidd or rabbitmq
5. sync the zoo repo and you'll receive this error (which could be improved, but not what this bug is about)

[vagrant@dev ~]$ pulp-admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

An internal error occurred on the Pulp server:

RequestException: POST request
on /pulp/api/v2/repositories/zoo/actions/sync/ failed with 500 - [Errno 111]
Connection refused

6. List the tasks with pulp-admin tasks list and observe that a task was created that can never be run!

Operations:  sync
Resources:   zoo (repository)
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     8e47a36a-2767-48a8-b6f6-e497d6f5cdec

The task above ^ will incorrectly reserve the worker it is assigned to and will incorrectly show as Waiting when it is never run. It will stay this way until the worker itself is restarted at which point the cleanup logic will make everything right again.

History

#2 Updated by ttereshc over 4 years ago

  • Priority changed from Normal to High
  • Sprint/Milestone set to 39
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes

#5 Updated by bmbouter over 4 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to bmbouter

#6 Updated by bmbouter over 4 years ago

  • Status changed from ASSIGNED to POST

#7 Updated by bmbouter over 4 years ago

  • Status changed from POST to MODIFIED

#9 Updated by pcreech over 4 years ago

  • Platform Release set to 2.13.2

#10 Updated by pcreech over 4 years ago

  • Status changed from MODIFIED to 5

#11 Updated by pcreech over 4 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE

#12 Updated by bmbouter almost 4 years ago

  • Sprint set to Sprint 20

#13 Updated by bmbouter almost 4 years ago

  • Sprint/Milestone deleted (39)

#14 Updated by bmbouter almost 3 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF