Project

Profile

Help

Maintenance: Planio will be observing a scheduled maintenance window this Sunday, November 10, 2024 from 20:00 UTC until 21:00 UTC to perform important network maintenance in our primary data center. Your Planio account will be unavailable for a few minutes during this maintenance window.

Issue #2930

closed

pulp_celerybeat keeps going offline

Added by dkliban@redhat.com over 7 years ago. Updated over 5 years ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
Start date:
Due date:
Estimated time:
Severity:
2. Medium
Version:
Platform Release:
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Sprint 23
Quarter:

Description

I see the following in my logs when I come back to my dev environment after a while.

[2017-07-23 16:39:06,658: ERROR/MainProcess] There are 0 pulp_celerybeat processes running. Pulp will not operate correctly without at least one pulp_celerybeat process running.
[2017-07-23 16:39:11,607: WARNING/MainProcess] Failover occurred: 'celerybeat@dev.example.com' is now the primary celerybeat instance
[2017-07-23 16:46:35,493: ERROR/MainProcess] Worker 'reserved_resource_worker_1@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:46:35,496: ERROR/MainProcess] The worker named reserved_resource_worker_1@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:46:36,547: ERROR/MainProcess] Worker 'reserved_resource_worker_2@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:46:36,547: ERROR/MainProcess] The worker named reserved_resource_worker_2@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:46:36,577: ERROR/MainProcess] Worker 'resource_manager@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:46:36,577: ERROR/MainProcess] The worker named resource_manager@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:46:36,603: ERROR/MainProcess] Worker 'celerybeat@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:46:36,603: ERROR/MainProcess] The worker named celerybeat@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:46:36,635: ERROR/MainProcess] There are 0 pulp_resource_manager processes running. Pulp will not operate correctly without at least one pulp_resource_mananger process running.
[2017-07-23 16:46:36,636: ERROR/MainProcess] There are 0 pulp_celerybeat processes running. Pulp will not operate correctly without at least one pulp_celerybeat process running.
[2017-07-23 16:46:41,608: WARNING/MainProcess] Failover occurred: 'celerybeat@dev.example.com' is now the primary celerybeat instance
[2017-07-23 16:49:23,592: ERROR/MainProcess] Worker 'reserved_resource_worker_1@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:49:23,592: ERROR/MainProcess] The worker named reserved_resource_worker_1@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:49:24,675: ERROR/MainProcess] Worker 'reserved_resource_worker_2@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:49:24,675: ERROR/MainProcess] The worker named reserved_resource_worker_2@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:49:24,686: ERROR/MainProcess] Worker 'resource_manager@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:49:24,687: ERROR/MainProcess] The worker named resource_manager@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:49:24,699: ERROR/MainProcess] Worker 'celerybeat@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:49:24,699: ERROR/MainProcess] The worker named celerybeat@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:49:24,724: ERROR/MainProcess] There are 0 pulp_resource_manager processes running. Pulp will not operate correctly without at least one pulp_resource_mananger process running.
[2017-07-23 16:49:24,724: ERROR/MainProcess] There are 0 pulp_celerybeat processes running. Pulp will not operate correctly without at least one pulp_celerybeat process running.
[2017-07-23 16:49:29,701: WARNING/MainProcess] Failover occurred: 'celerybeat@dev.example.com' is now the primary celerybeat instance
[2017-07-23 16:52:01,564: ERROR/MainProcess] Worker 'reserved_resource_worker_1@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:52:01,564: ERROR/MainProcess] The worker named reserved_resource_worker_1@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:52:02,606: ERROR/MainProcess] Worker 'reserved_resource_worker_2@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:52:02,607: ERROR/MainProcess] The worker named reserved_resource_worker_2@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:52:02,628: ERROR/MainProcess] Worker 'resource_manager@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:52:02,629: ERROR/MainProcess] The worker named resource_manager@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:52:02,642: ERROR/MainProcess] Worker 'celerybeat@dev.example.com' has gone missing, removing from list of workers
[2017-07-23 16:52:02,642: ERROR/MainProcess] The worker named celerybeat@dev.example.com is missing. Canceling the tasks in its queue.
[2017-07-23 16:52:02,669: ERROR/MainProcess] There are 0 pulp_resource_manager processes running. Pulp will not operate correctly without at least one pulp_resource_mananger process running.
[2017-07-23 16:52:02,669: ERROR/MainProcess] There are 0 pulp_celerybeat processes running. Pulp will not operate correctly without at least one pulp_celerybeat process running.
[2017-07-23 16:52:07,634: WARNING/MainProcess] Failover occurred: 'celerybeat@dev.example.com' is now the primary celerybeat instance
Actions #1

Updated by ttereshc over 7 years ago

  • Sprint/Milestone set to 42
  • Triaged changed from No to Yes
Actions #2

Updated by fdobrovo over 7 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to fdobrovo
Actions #3

Updated by dkliban@redhat.com over 7 years ago

  • Status changed from ASSIGNED to CLOSED - WORKSFORME

This issue has stopped presenting itself. It probably has to do with my laptop going into hibernation and the network going offline.

Actions #4

Updated by fdobrovo over 7 years ago

  • Status changed from CLOSED - WORKSFORME to CLOSED - WONTFIX

This happens whenever there is a relative forward time jump. This can happen if the virtual machine is temporally suspended due to host hibernation.

Actions #5

Updated by bmbouter over 6 years ago

  • Sprint set to Sprint 23
Actions #6

Updated by bmbouter over 6 years ago

  • Sprint/Milestone deleted (42)
Actions #7

Updated by daviddavis over 5 years ago

  • Sprint/Milestone set to 3.0.0
Actions #8

Updated by bmbouter over 5 years ago

  • Tags deleted (Pulp 3)

Also available in: Atom PDF