Project

Profile

Help

Issue #2930

pulp_celerybeat keeps going offline

Added by dkliban@redhat.com about 2 years ago. Updated 6 months ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Severity:
2. Medium
Version:
Platform Release:
Blocks Release:
OS:
Backwards Incompatible:
No
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 23

Description

I see the following in my logs when I come back to my dev environment after a while.

[2017-07-23 16:39:06,658: ERROR/MainProcess] There are 0 pulp_celerybeat processes running. Pulp will not operate correctly without at least one pulp_celerybeat process running.
[2017-07-23 16:39:11,607: WARNING/MainProcess] Failover occurred: 'celerybeat@dev.example.com' is now the primary celerybeat instance
[2017-07-23 16:46:35,493: ERROR/MainProcess] Worker 'reserved_resource_worker_1@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:46:35,496: ERROR/MainProcess] The worker named reserved_resource_worker_1@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:46:36,547: ERROR/MainProcess] Worker 'reserved_resource_worker_2@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:46:36,547: ERROR/MainProcess] The worker named reserved_resource_worker_2@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:46:36,577: ERROR/MainProcess] Worker 'resource_manager@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:46:36,577: ERROR/MainProcess] The worker named resource_manager@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:46:36,603: ERROR/MainProcess] Worker 'celerybeat@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:46:36,603: ERROR/MainProcess] The worker named celerybeat@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:46:36,635: ERROR/MainProcess] There are 0 pulp_resource_manager processes running. Pulp will not operate correctly without at least one pulp_resource_mananger process running.
[2017-07-23 16:46:36,636: ERROR/MainProcess] There are 0 pulp_celerybeat processes running. Pulp will not operate correctly without at least one pulp_celerybeat process running.
[2017-07-23 16:46:41,608: WARNING/MainProcess] Failover occurred: 'celerybeat@dev.example.com' is now the primary celerybeat instance
[2017-07-23 16:49:23,592: ERROR/MainProcess] Worker 'reserved_resource_worker_1@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:49:23,592: ERROR/MainProcess] The worker named reserved_resource_worker_1@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:49:24,675: ERROR/MainProcess] Worker 'reserved_resource_worker_2@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:49:24,675: ERROR/MainProcess] The worker named reserved_resource_worker_2@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:49:24,686: ERROR/MainProcess] Worker 'resource_manager@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:49:24,687: ERROR/MainProcess] The worker named resource_manager@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:49:24,699: ERROR/MainProcess] Worker 'celerybeat@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:49:24,699: ERROR/MainProcess] The worker named celerybeat@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:49:24,724: ERROR/MainProcess] There are 0 pulp_resource_manager processes running. Pulp will not operate correctly without at least one pulp_resource_mananger process running.
[2017-07-23 16:49:24,724: ERROR/MainProcess] There are 0 pulp_celerybeat processes running. Pulp will not operate correctly without at least one pulp_celerybeat process running.
[2017-07-23 16:49:29,701: WARNING/MainProcess] Failover occurred: 'celerybeat@dev.example.com' is now the primary celerybeat instance
[2017-07-23 16:52:01,564: ERROR/MainProcess] Worker 'reserved_resource_worker_1@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:52:01,564: ERROR/MainProcess] The worker named reserved_resource_worker_1@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:52:02,606: ERROR/MainProcess] Worker 'reserved_resource_worker_2@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:52:02,607: ERROR/MainProcess] The worker named reserved_resource_worker_2@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:52:02,628: ERROR/MainProcess] Worker 'resource_manager@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:52:02,629: ERROR/MainProcess] The worker named resource_manager@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:52:02,642: ERROR/MainProcess] Worker 'celerybeat@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:52:02,642: ERROR/MainProcess] The worker named celerybeat@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:52:02,669: ERROR/MainProcess] There are 0 pulp_resource_manager processes running. Pulp will not operate correctly without at least one pulp_resource_mananger process running.
[2017-07-23 16:52:02,669: ERROR/MainProcess] There are 0 pulp_celerybeat processes running. Pulp will not operate correctly without at least one pulp_celerybeat process running.
[2017-07-23 16:52:07,634: WARNING/MainProcess] Failover occurred: 'celerybeat@dev.example.com' is now the primary celerybeat instance

History

#1 Updated by ttereshc about 2 years ago

  • Sprint/Milestone set to 42
  • Triaged changed from No to Yes

#2 Updated by fdobrovo about 2 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fdobrovo

#3 Updated by dkliban@redhat.com about 2 years ago

  • Status changed from ASSIGNED to CLOSED - WORKSFORME

This issue has stopped presenting itself. It probably has to do with my laptop going into hibernation and the network going offline.

#4 Updated by fdobrovo about 2 years ago

  • Status changed from CLOSED - WORKSFORME to CLOSED - WONTFIX

This happens whenever there is a relative forward time jump. This can happen if the virtual machine is temporally suspended due to host hibernation.

#5 Updated by bmbouter over 1 year ago

  • Sprint set to Sprint 23

#6 Updated by bmbouter over 1 year ago

  • Sprint/Milestone deleted (42)

#7 Updated by daviddavis 6 months ago

  • Sprint/Milestone set to 3.0

#8 Updated by bmbouter 6 months ago

  • Tags deleted (Pulp 3)

Please register to edit this issue

Also available in: Atom PDF