Story #1939
closed
As a user, I would like to be able to profile Pulp tasks
Status:
CLOSED - CURRENTRELEASE
Description
Problem Overview¶
Often, users (both developers and people out in the "real world") want to be able to figure out why a task is taking a long time to complete. What bits are taking the majority of the time? It is possible to answer this question with profiling tools. Python's standard library contains ``profile`` and ``cProfile``. It would nice if I could flip a configuration setting and have every task Pulp dispatches be profiled.
Implementation Plan¶
It might be possible (and easy, even) to add to the task decorator we use on functions to check the server config for a setting, ``profile_tasks``. If true, it wraps the task function with cProfile and sends it on its merry way. There are, of course, other ways we could do this, but that's just one way that came to mind.
@jcline, FYI dkliban had put this together: http://pulp.readthedocs.io/en/latest/dev-guide/debugging.html
We considered doing something similar to this story. We still can, but our reasoning not to was that a developer would need to read the report, so we expected a developer could add the 2-3 lines of import statements needed to generate the report.
What do you think?
I do normally add the necessary lines, but I find it onerous. I also just encountered a situation where a user was experiencing slow tasks. Instead of being able to say "Hey, flip on profiling and send me these files", I had to either walk them through writing Python or (as was the case yesterday) get access to the system myself.
I hadn't considered the remote debugging case; that makes sense, let's do it.
- Status changed from NEW to POST
- Assignee set to ehelms@redhat.com
- Sprint/Milestone set to 30
- Status changed from POST to MODIFIED
- % Done changed from 0 to 100
- Platform Release set to 2.12.0
- Status changed from MODIFIED to 5
- Status changed from 5 to ASSIGNED
I discovered this feature was not working as implemented so I'm pulling back to assigned.
- The code was using a slightly different spelling of the config value so it wasn't finding it
- The config value was being treated as a boolean when it was a string
- It did not provide directory management so it fails unless the directory is manually created
I'm making a PR now to fix all of these things.
Also the quoted section of docs weren't rendering as expected. I'm fixing that too.
- Status changed from ASSIGNED to POST
- Status changed from POST to MODIFIED
- Status changed from MODIFIED to 5
- Status changed from 5 to CLOSED - CURRENTRELEASE
- Sprint/Milestone deleted (
30)
Also available in: Atom
PDF
.
closes #1939: Enable cProfile of tasks
Adds the ability to turn on cProfiles of individual Pulp tasks and dumping of the tasks into a directory. This can have an impact on performance. The trade off is that users could investigate individual tasks or provide these cProfiles to developers to help detect anomalies or systematic issues.
https://pulp.plan.io/issues/1939