Story #4689
Updated by bmbouter over 4 years ago
## Motivation Pulp as a multi-process system needs to make debugging easy. Currently it is hard to: 1. Identify which log statements are related to a specific API request 2. Filter out all of those that aren't 3. Do this across multiple processes Additionally, there is a use case where an external system wants to provide the correlation id, e.g. Katello wants to submit the correlation ID as part of the request. ## Solution Use [django-cid](https://django-correlation-id.readthedocs.io/en/latest/) by default, and document that it's available for users to use. ## Implementation ### Enabling in Pulp itself 1. Add django-cid as an installed app 2. Reconfigure the default Django logger config in [settings.py](https://github.com/pulp/pulpcore/blob/master/pulpcore/app/settings.py#L178-L190) to incorporate the CID 3. Add user documentation identifying the types of things they can additionally configure 4. Add a test to assert Pulp viewset logs contain an autogenerated CID ### Enabling in the App 1. Add a test to assert content app logs contain an autogenerated CID ### Enabling in Tasking System 1. *Dispatching the CID without the caller being involved:* Have the [`_queue_reserved_task`](https://github.com/pulp/pulpcore/blob/b94abd64d76ea4554e6750ff38ce458eaa888cc8/pulpcore/tasking/tasks.py#L51) [`_queue_reserved_task`](https://github.com/pulp/pulpcore/blob/b94abd64d76ea4554e6750ff38ce458eaa888cc8/pulpcore/tasking/tasks.py#L162) method expect a new key named `_pulp_logging_cid` in the existing `inner_kwargs` kwarg. The [`enqueue_with_reservation`](https://github.com/pulp/pulpcore/blob/a315ce7a6d3575da4308298081378952764333cb/pulpcore/tasking/tasks.py#L162) method will transparently add the `_pulp_logging_cid` in the existing `inner_kwargs` kwarg. `enqueue_with_reservation` should get the value of `_pulp_logging_cid` at dispatch time using [`get_cid`](https://django-correlation-id.readthedocs.io/en/latest/api.html#cid.locals.get_cid). The [`enqueue_with_reservation`](https://github.com/pulp/pulpcore/blob/a315ce7a6d3575da4308298081378952764333cb/pulpcore/tasking/tasks.py#L162) method will not have its signature change. 2. *Have the CID log on the resource manager itself:* Have the [`_queue_reserved_task`](https://github.com/pulp/pulpcore/blob/b94abd64d76ea4554e6750ff38ce458eaa888cc8/pulpcore/tasking/tasks.py#L51) method call [`set_cid()`](https://django-correlation-id.readthedocs.io/en/latest/api.html#cid.locals.set_cid) with the value of the new `_pulp_logging_cid` key in the existing `inner_kwargs` kwargs. This will enable the correlation ID in the resource_manager logs for that task also. 3. *Have all tasks also use the CID:* Have the [`perform_job`](https://github.com/pulp/pulpcore/blob/master/pulpcore/tasking/worker.py#L75) call [`set_cid()`](https://django-correlation-id.readthedocs.io/en/latest/api.html#cid.locals.set_cid) with the value of the `_pulp_logging_cid` kwarg. Then since we don't want to actually pass it to the RQ task itself, pop it off of the kwargs for the task. This will enable the correlation ID in all pulp task logs without them having to change their signatures.
.