Project

Profile

Help

Task #808

Set heartbeat timeout for pulp_workers and pulp_resource_manager to 30 seconds

Added by bmbouter over 6 years ago. Updated 8 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
High
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Platform Release:
2.7.1
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Pulp 2
Sprint:
April 2015
Quarter:

Description

pulp_workers and pulp_resource_manager both use vanilla Celery heartbeats. I believe the default is 2 seconds. We should adjust that to 30 seconds since we aren't benefiting from such a quick heartbeat interval. pulp_resource_manager and pulp_workers should use the heartbeat-interval command line option to set this.

Deliverables:

  • Add in command line option for pulp_workers to use --heartbeat-interval
  • Add in command line option for pulp_resource_manager to use --heartbeat-interval
  • ensure both work on EL6 and EL7 because they use different init systems (systemd vs upstart)
  • Verify that traffic to mongodb decreases for an idle system (consider using iotop)

Associated revisions

Revision c896c2ad View on GitHub
Added by cduryee over 6 years ago

set heartbeat interval to 30s

The heartbeat interval for celery was previously set to the default of 2 seconds. This caused the workers collection to be updated in mongo every two seconds as well, even on an idle system.

Instead, set the heartbeat interval to 30 seconds. This should be sufficient for worker status updates.

fixes #808

Revision c896c2ad View on GitHub
Added by cduryee over 6 years ago

set heartbeat interval to 30s

The heartbeat interval for celery was previously set to the default of 2 seconds. This caused the workers collection to be updated in mongo every two seconds as well, even on an idle system.

Instead, set the heartbeat interval to 30 seconds. This should be sufficient for worker status updates.

fixes #808

History

#1 Updated by mhrivnak over 6 years ago

  • Description updated (diff)
  • Tags Groomed added

#2 Updated by cduryee over 6 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to cduryee

#3 Updated by cduryee over 6 years ago

  • Status changed from ASSIGNED to POST
  • Sprint/Milestone set to 15

#4 Updated by cduryee over 6 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#5 Updated by bmbouter over 6 years ago

  • Groomed set to Yes
  • Tags deleted (Groomed)

#6 Updated by bmbouter over 6 years ago

  • Sprint Candidate set to Yes
  • Tags deleted (Sprint Candidate)

#7 Updated by rbarlow over 5 years ago

  • Status changed from MODIFIED to CLOSED - CURRENTRELEASE
  • Platform Release set to 2.7.2

#8 Updated by rbarlow over 5 years ago

  • Platform Release changed from 2.7.2 to 2.7.1

#9 Updated by bmbouter over 3 years ago

  • Sprint set to April 2015

#10 Updated by bmbouter over 3 years ago

  • Sprint/Milestone deleted (15)

#11 Updated by bmbouter over 2 years ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF