Project

Profile

Help

Issue #2516

closed

worker heartbeat log statement mishandles timezone

Added by mhrivnak over 5 years ago. Updated about 3 years ago.

Status:
CLOSED - NOTABUG
Priority:
Normal
Assignee:
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
Estimated time:
Severity:
1. Low
Version:
2.8.7
Platform Release:
2.13.0
OS:
Triaged:
Yes
Groomed:
No
Sprint Candidate:
No
Tags:
Easy Fix, Pulp 2
Sprint:
Sprint 14
Quarter:

Description

From bugzilla:

Dec 22 14:03:55 pmoravec-sat62-rhel6 pulp: pulp.server.async.worker_watcher:DEBUG: 'worker-heartbeat' sent at time 2016-12-22 14:03:55.439300 from reserved_resource_worker-3@pmoravec-sat62-rhel6.gsslab.brq.redhat.com, received at time: 2016-12-22 13:03:55.450517

Saying that:
- worker sent heartbeat at 14:03
- the heartbeat was received at 13:03

The machine was set to GTM+1.

The code is here:

https://github.com/pulp/pulp/blob/pulp-2.11.0-1/server/pulp/server/async/worker_watcher.py#L27

Actions #1

Updated by bmbouter over 5 years ago

We generate our own timestamps and only use the worker name from the heartbeat. Instead of logging the heartbeat we receive, we should create our own heartbeat log line with the worker name and timestamp that we are using (the one we generate).

Actions #2

Updated by bizhang over 5 years ago

  • Sprint/Milestone set to 31
  • Severity changed from 2. Medium to 1. Low
  • Triaged changed from No to Yes
Actions #3

Updated by dalley over 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley
Actions #4

Updated by mhrivnak over 5 years ago

  • Sprint/Milestone changed from 31 to 32
Actions #5

Updated by dalley over 5 years ago

  • Status changed from ASSIGNED to NEW
Actions #6

Updated by dalley over 5 years ago

  • Assignee deleted (dalley)
Actions #7

Updated by dalley over 5 years ago

Unassigning myself for grooming - jumped the gun a bit

Actions #8

Updated by dalley over 5 years ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to dalley

Assigning it to myself - will be resolved by the fix for Issue 2519 - https://github.com/pulp/pulp/pull/2922

No further effort needed

Actions #10

Updated by dalley over 5 years ago

  • Status changed from ASSIGNED to POST

Added by dalley over 5 years ago

Revision fd19f890

Workers write their own hearbeat records to database.

All workers will write their own records to the database instead of relying on pulp_celerybeat to do so for them using celery heartbeats.

This patch makes use of the Consumer blueprint that celery runs at the start time of a worker. An extra boot step has been added which sets a timer to periodically update the worker record in the database.

http://docs.celeryproject.org/en/master/userguide/extending.html https://groups.google.com/d/msg/celery-users/3fs0ocREYqw/C7U1lCAp56sJ

closes #2519 https://pulp.plan.io/issues/2519 closes #2516 https://pulp.plan.io/issues/2516

Added by dalley over 5 years ago

Revision fd19f890

Workers write their own hearbeat records to database.

All workers will write their own records to the database instead of relying on pulp_celerybeat to do so for them using celery heartbeats.

This patch makes use of the Consumer blueprint that celery runs at the start time of a worker. An extra boot step has been added which sets a timer to periodically update the worker record in the database.

http://docs.celeryproject.org/en/master/userguide/extending.html https://groups.google.com/d/msg/celery-users/3fs0ocREYqw/C7U1lCAp56sJ

closes #2519 https://pulp.plan.io/issues/2519 closes #2516 https://pulp.plan.io/issues/2516

Actions #11

Updated by dalley over 5 years ago

  • Status changed from POST to MODIFIED
Actions #13

Updated by pthomas@redhat.com over 5 years ago

Manual verification as per smash issue #513

Actions #14

Updated by semyers about 5 years ago

  • Platform Release set to 2.13.0
Actions #15

Updated by pcreech about 5 years ago

  • Status changed from MODIFIED to 5
Actions #16

Updated by pcreech about 5 years ago

  • Status changed from 5 to CLOSED - CURRENTRELEASE
Actions #17

Updated by dalley over 4 years ago

  • Status changed from CLOSED - CURRENTRELEASE to CLOSED - WONTFIX

See discussion in the associated bugzilla. Revisited due to FailedQA, but we recognized that it is perfectly fine to remain the way it is currently, and isn't an issue.

Actions #18

Updated by dalley over 4 years ago

  • Status changed from CLOSED - WONTFIX to CLOSED - NOTABUG

I suppose this should actually be "NOTABUG" instead of "WONTFIX"

Actions #19

Updated by bmbouter about 4 years ago

  • Sprint set to Sprint 16
Actions #20

Updated by bmbouter about 4 years ago

  • Sprint changed from Sprint 16 to Sprint 14
Actions #21

Updated by bmbouter about 4 years ago

  • Sprint/Milestone deleted (32)
Actions #22

Updated by bmbouter about 3 years ago

  • Tags Pulp 2 added

Also available in: Atom PDF