Project

Profile

Help

Story #4689

As a user, Pulp logs have correlation ids in the logs of all services

Added by dkliban@redhat.com about 1 year ago. Updated about 8 hours ago.

Status:
ASSIGNED
Priority:
Normal
Assignee:
Category:
-
Start date:
Due date:
% Done:

0%

Platform Release:
Blocks Release:
Groomed:
Yes
Sprint Candidate:
Yes
Tags:
Katello
QA Contact:
Complexity:
Smash Test:
Sprint:
Sprint 74

Description

Motivation

Pulp as a multi-process system needs to make debugging easy. Currently it is hard to:

  1. Identify which log statements are related to a specific API request
  2. Filter out all of those that aren't
  3. Do this across multiple processes

Additionally, there is a use case where an external system wants to provide the correlation id, e.g. Katello wants to submit the correlation ID as part of the request.

Solution

Use django-cid by default, and document that it's available for users to use.

Implementation

Enabling in Pulp itself

  1. Add django-cid as an installed app
  2. Reconfigure the default Django logger config in settings.py to incorporate the CID
  3. Add user documentation identifying the types of things they can additionally configure
  4. Add a test to assert Pulp viewset logs contain an autogenerated CID

Enabling in the App

  1. Add a test to assert content app logs contain an autogenerated CID

Enabling in Tasking System

  1. Dispatching the CID without the caller being involved: Have the _queue_reserved_task method expect a new key named _pulp_logging_cid in the existing inner_kwargs kwarg. The enqueue_with_reservation method will transparently add the _pulp_logging_cid in the existing inner_kwargs kwarg by getting the value of _pulp_logging_cid at dispatch time using get_cid. The enqueue_with_reservation method will not have its signature change.

  2. Have the CID log on the resource manager itself: Have the _queue_reserved_task method call set_cid() with the value of the new _pulp_logging_cid key in the existing inner_kwargs kwargs. This will enable the correlation ID in the resource_manager logs for that task also.

  3. Have all tasks also use the CID: Have the perform_job call set_cid() with the value of the _pulp_logging_cid kwarg. Then since we don't want to actually pass it to the RQ task itself, pop it off of the kwargs for the task. This will enable the correlation ID in all pulp task logs without them having to change their signatures.


Related issues

Related to Pulp - Task #3289: Support for X-Correlation-ID HTTP REST header CLOSED - WONTFIX Actions

History

#1 Updated by bmbouter about 1 year ago

  • Tags deleted (Pulp 3)

#2 Updated by jsherril@redhat.com 11 months ago

  • Tags Katello-P3 added

#3 Updated by bmbouter 5 months ago

  • Related to Task #3289: Support for X-Correlation-ID HTTP REST header added

#5 Updated by ggainey 28 days ago

  • Priority changed from Normal to Low

#6 Updated by ggainey 28 days ago

  • Tags Katello added
  • Tags deleted (Katello-P3)

#7 Updated by bmbouter 25 days ago

  • Subject changed from As a user, I can configure pulp to accept a correlation id header to As a user, Pulp logs have correlation ids in the logs of all services
  • Description updated (diff)

#8 Updated by bmbouter 25 days ago

  • Checklist item deleted (documentation on how to enable and configure)

#9 Updated by daviddavis 15 days ago

For this kwarg _pulp_logging_cid, will the code calling enqueue_with_reservation have to manually set this if it wants task logging to use this CID?

#10 Updated by bmbouter 15 days ago

daviddavis wrote:

For this kwarg _pulp_logging_cid, will the code calling enqueue_with_reservation have to manually set this if it wants task logging to use this CID?

The caller of enqueue_with_reservation will not have to specify the CID, the CID will automatically and transparently be added by enqueue_with_reservation itself. The signature of enqueue_with_reservation won't be adjusted with this change so rolling it out will require no other code to change. Also when rolled out it, will be enabled automatically on all tasks.

#11 Updated by bmbouter 15 days ago

  • Description updated (diff)

#12 Updated by bmbouter 15 days ago

  • Description updated (diff)

#13 Updated by bmbouter 15 days ago

  • Description updated (diff)

#14 Updated by daviddavis 15 days ago

  • Groomed changed from No to Yes
  • Sprint Candidate changed from No to Yes

#15 Updated by rchan 8 days ago

  • Sprint set to Sprint 74

#16 Updated by daviddavis about 8 hours ago

  • Status changed from NEW to ASSIGNED
  • Assignee set to daviddavis
  • Priority changed from Low to Normal

Please register to edit this issue

Also available in: Atom PDF