Project

Profile

Help

Task #5460

Investigate sync performance

Added by bmbouter 28 days ago. Updated 21 days ago.

Status:
NEW
Priority:
Normal
Assignee:
-
Category:
-
Sprint/Milestone:
-
Start date:
Due date:
% Done:

0%

Platform Release:
Blocks Release:
Backwards Incompatible:
No
Groomed:
No
Sprint Candidate:
Yes
Tags:
QA Contact:
Complexity:
Smash Test:
Verified:
No
Verification Required:
No
Sprint:

Description

The performance testing revealed two concerns that need to be investigated. These are summarized in the report from the performance team here: https://github.com/pulp/pulpproject.org/pull/214/files#diff-50f8bac543f30a639b3cf46d4f415b2bR105

Using the tests moved to pulp_file with https://pulp.plan.io/issues/5458 first 1) reproduce the issue and 2) determine where the time is being spent with cprofile capturing.


Related issues

Blocked by File Support - Task #5458: Port the performance-tests to pulp_file POST Actions

History

#1 Updated by bmbouter 28 days ago

  • Blocked by Task #5458: Port the performance-tests to pulp_file added

#2 Updated by dalley 28 days ago

I'm going to plug a couple of other tools that should be evaluated by the person who does this work. It might make your life easier.

Py-Spy
https://github.com/benfred/py-spy

This is sort of like cProfile, except that instead of tracing and recording the execution of every operation, it takes snapshots of the interpreter X times per second to see what functions are running for the most time. Unlike cProfile it does not influence the runtime performance of the operation -- running cProfile usually makes everything take several times longer than it otherwise would have which is painful if you have to re-run things frequently. It also has a nice live ncurses interface that you can use to sort and filter on certain parameters. And it can generate flamegraphs which are not quite as readable as cProfile reports but are servicable. You don't have to set it up beforehand, you can just point it at an already running PID.

pghero
https://github.com/ankane/pghero

If it turns out that the performance issues are in the database queries being made then pghero can help figure out why the queries are so inefficient. There's a nice live demo link here https://pghero.dokkuapp.com/datakick

pgadmin
https://www.pgadmin.org/

Big brother of pghero

#3 Updated by fabricio.aguiar 21 days ago

  • Sprint Candidate changed from No to Yes

Please register to edit this issue

Also available in: Atom PDF