Project

Profile

Help

Task #2971

closed

enable celery workers to log at levels below WARN

Added by mhrivnak over 6 years ago. Updated almost 5 years ago.

Status:
CLOSED - WONTFIX
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
Start date:
Due date:
% Done:

0%

Estimated time:
Platform Release:
Groomed:
No
Sprint Candidate:
No
Tags:
Sprint:
Quarter:

Description

Celery workers set their own log level somehow. You can supply one via a celery setting, either on the command line or as an environment variable.

http://docs.celeryproject.org/en/v4.1.0/reference/celery.bin.worker.html#cmdoption-celery-worker-l

It defaults to WARN. I was able to get the workers to log at info by adding "--loglevel=INFO" to the worker unit file. That is probably not the best approach though. It is desirable to have a setting that can enable DEBUG logging across all of Pulp or not.

I think in Pulp 2 there was a mechanism for overriding whatever log level celery set, but I'm not entirely sure. This task is to figure out a reasonable approach for setting the log level across pulp and to make it happen.


Related issues

Is duplicate of Pulp - Issue #2906: Level INFO messages not being logged in worker processes.CLOSED - CURRENTRELEASEbizhangActions
Actions #1

Updated by bizhang over 6 years ago

There is a celery setting to stop it from hijacking the root logger: http://docs.celeryproject.org/en/master/userguide/configuration.html#worker-hijack-root-logger

Actions #2

Updated by bizhang over 6 years ago

Another issue is that celery does no use the provided formatter when it hijacks logging.
Because of this I would recommend we stop celery from hijacking the root logger instead of re-configuring celery logging with the correct formatter in addition to the correct log level.

Actions #3

Updated by bizhang over 6 years ago

I looks that the worker_hijack_root_logger only stops celery from hijacking the handlers and not the loglevel. [0]

The way to stop if from hijacking everything is to use the celery setup_logging signal as per:
https://github.com/celery/celery/issues/2509#issuecomment-153936466

[0] https://github.com/celery/celery/blob/29b62aed17e99980668984446923089f17d300c0/celery/app/log.py#L110

Actions #4

Updated by mhrivnak over 6 years ago

  • Is duplicate of Issue #2906: Level INFO messages not being logged in worker processes. added
Actions #5

Updated by mhrivnak over 6 years ago

  • Status changed from NEW to CLOSED - DUPLICATE
Actions #6

Updated by dalley about 6 years ago

  • Status changed from CLOSED - DUPLICATE to NEW
Actions #7

Updated by dalley almost 6 years ago

  • Status changed from NEW to CLOSED - WONTFIX

No longer applicable w/ switch away from Celery

Actions #8

Updated by daviddavis almost 5 years ago

  • Sprint/Milestone set to 3.0.0
Actions #9

Updated by bmbouter almost 5 years ago

  • Tags deleted (Pulp 3)

Also available in: Atom PDF