Project

Profile

Help

Story #1939

As a user, I would like to be able to profile Pulp tasks

Added by jcline@redhat.com almost 4 years ago. Updated 11 months ago.

Status:
CLOSED - CURRENTRELEASE
Priority:
Normal
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

100%

Platform Release:
2.12.0
Blocks Release:
Backwards Incompatible:
No
Groomed:
No
Sprint Candidate:
No
Tags:
Pulp 2
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:
Sprint 12

Description

Problem Overview

Often, users (both developers and people out in the "real world") want to be able to figure out why a task is taking a long time to complete. What bits are taking the majority of the time? It is possible to answer this question with profiling tools. Python's standard library contains ``profile`` and ``cProfile``. It would nice if I could flip a configuration setting and have every task Pulp dispatches be profiled.

Implementation Plan

It might be possible (and easy, even) to add to the task decorator we use on functions to check the server config for a setting, ``profile_tasks``. If true, it wraps the task function with cProfile and sends it on its merry way. There are, of course, other ways we could do this, but that's just one way that came to mind.

Associated revisions

Revision 07202a9b View on GitHub
Added by Eric D Helms about 3 years ago

closes #1939: Enable cProfile of tasks

Adds the ability to turn on cProfiles of individual Pulp tasks and dumping of the tasks into a directory. This can have an impact on performance. The trade off is that users could investigate individual tasks or provide these cProfiles to developers to help detect anomalies or systematic issues.

https://pulp.plan.io/issues/1939

Revision 07202a9b View on GitHub
Added by Eric D Helms about 3 years ago

closes #1939: Enable cProfile of tasks

Adds the ability to turn on cProfiles of individual Pulp tasks and dumping of the tasks into a directory. This can have an impact on performance. The trade off is that users could investigate individual tasks or provide these cProfiles to developers to help detect anomalies or systematic issues.

https://pulp.plan.io/issues/1939

Revision 7e5d817e View on GitHub
Added by bmbouter about 3 years ago

Merge pull request #2882 from ehelms/fix-1939

Fixes #1939: Enable cProfile of tasks

Revision 7e5d817e View on GitHub
Added by bmbouter about 3 years ago

Merge pull request #2882 from ehelms/fix-1939

Fixes #1939: Enable cProfile of tasks

Revision b4db6138 View on GitHub
Added by bmbouter about 3 years ago

Fixes the cProfile Feature

This feature was introduced in 2.12, and is being fixed prior to the release of 2.12.0. This fix includes:

  • Updates the server.conf which used enable instead of enabled
  • Updates the config file checking which treated received a str when it expected a boolean.
  • Provides automatic directory creation.
  • Documentation updates and fix for blockquote sections which did not render correctly.

https://pulp.plan.io/issues/1939 closes #1939

Revision b4db6138 View on GitHub
Added by bmbouter about 3 years ago

Fixes the cProfile Feature

This feature was introduced in 2.12, and is being fixed prior to the release of 2.12.0. This fix includes:

  • Updates the server.conf which used enable instead of enabled
  • Updates the config file checking which treated received a str when it expected a boolean.
  • Provides automatic directory creation.
  • Documentation updates and fix for blockquote sections which did not render correctly.

https://pulp.plan.io/issues/1939 closes #1939

History

#1 Updated by bmbouter almost 4 years ago

@jcline, FYI dkliban had put this together: http://pulp.readthedocs.io/en/latest/dev-guide/debugging.html

We considered doing something similar to this story. We still can, but our reasoning not to was that a developer would need to read the report, so we expected a developer could add the 2-3 lines of import statements needed to generate the report.

What do you think?

#2 Updated by jcline@redhat.com almost 4 years ago

I do normally add the necessary lines, but I find it onerous. I also just encountered a situation where a user was experiencing slow tasks. Instead of being able to say "Hey, flip on profiling and send me these files", I had to either walk them through writing Python or (as was the case yesterday) get access to the system myself.

#3 Updated by bmbouter almost 4 years ago

I hadn't considered the remote debugging case; that makes sense, let's do it.

#5 Updated by bmbouter about 3 years ago

  • Status changed from NEW to POST
  • Assignee set to ehelms@redhat.com
  • Sprint/Milestone set to 30

@ehelms put in a PR on this so I'm setting him as assignee.

PR available at: https://github.com/pulp/pulp/pull/2882/

#6 Updated by Anonymous about 3 years ago

  • Status changed from POST to MODIFIED
  • % Done changed from 0 to 100

#7 Updated by semyers about 3 years ago

  • Platform Release set to 2.12.0

#8 Updated by semyers about 3 years ago

  • Status changed from MODIFIED to ON_QA

#9 Updated by bmbouter about 3 years ago

  • Status changed from ON_QA to ASSIGNED

I discovered this feature was not working as implemented so I'm pulling back to assigned.

  • The code was using a slightly different spelling of the config value so it wasn't finding it
  • The config value was being treated as a boolean when it was a string
  • It did not provide directory management so it fails unless the directory is manually created

I'm making a PR now to fix all of these things.

#10 Updated by bmbouter about 3 years ago

Also the quoted section of docs weren't rendering as expected. I'm fixing that too.

#11 Updated by bmbouter about 3 years ago

  • Smash Test set to 503

#12 Updated by bmbouter about 3 years ago

  • Status changed from ASSIGNED to POST

#13 Updated by bmbouter about 3 years ago

  • Status changed from POST to MODIFIED

#14 Updated by semyers about 3 years ago

  • Status changed from MODIFIED to ON_QA

#15 Updated by semyers about 3 years ago

  • Status changed from ON_QA to CLOSED - CURRENTRELEASE

#16 Updated by bmbouter almost 2 years ago

  • Sprint set to Sprint 12

#17 Updated by bmbouter almost 2 years ago

  • Sprint/Milestone deleted (30)

#18 Updated by bmbouter 11 months ago

  • Tags Pulp 2 added

Please register to edit this issue

Also available in: Atom PDF