Project

Profile

Help

Issue #2770

closed

Tasks stuck in waiting state if received while qpidd is down

Added by bmbouter over 7 years ago. Updated over 5 years ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
3. High
Version:
Platform Release:
2.13.2
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
Sprint:
Sprint 20
Quarter:

Description

To reproduce:

1. Start a normal, healthy Pulp system.
2. sync the zoo repo to make sure your system is health. Ensure this completes
3. Observe that there are 0 currently running tasks using pulp-admin tasks list
4. Stop qpidd or rabbitmq
5. sync the zoo repo and you'll receive this error (which could be improved, but not what this bug is about)

[vagrant@dev ~]$ pulp-admin rpm repo sync run --repo-id zoo
+----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

An internal error occurred on the Pulp server:

RequestException: POST request
on /pulp/api/v2/repositories/zoo/actions/sync/ failed with 500 - [Errno 111]
Connection refused

6. List the tasks with pulp-admin tasks list and observe that a task was created that can never be run!

Operations:  sync
Resources:   zoo (repository)
State:       Waiting
Start Time:  Unstarted
Finish Time: Incomplete
Task Id:     8e47a36a-2767-48a8-b6f6-e497d6f5cdec

The task above ^ will incorrectly reserve the worker it is assigned to and will incorrectly show as Waiting when it is never run. It will stay this way until the worker itself is restarted at which point the cleanup logic will make everything right again.

Actions #2

Updated by ttereshc over 7 years ago

  • Priority changed from Normal to High
  • Sprint/Milestone set to 39
  • Severity changed from 2. Medium to 3. High
  • Triaged changed from No to Yes
Actions #5

Updated by bmbouter over 7 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to bmbouter
Actions #6

Updated by bmbouter over 7 years ago

  • Status changed from ASSIGNED to POST
Actions #7

Updated by bmbouter over 7 years ago

  • Status changed from POST to MODIFIED
Actions #9

Updated by pcreech over 7 years ago

  • Platform Release set to 2.13.2
Actions #10

Updated by pcreech over 7 years ago

  • Status changed from MODIFIED to 5
Actions #11

Updated by pcreech over 7 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #12

Updated by bmbouter over 6 years ago

  • Sprint set to Sprint 20
Actions #13

Updated by bmbouter over 6 years ago

  • Sprint/Milestone deleted (39)
Actions #14

Updated by bmbouter over 5 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF