Task #5460
closed
Investigate sync performance
0%
Description
The performance testing revealed two concerns that need to be investigated. These are summarized in the report from the performance team here: https://github.com/pulp/pulpproject.org/pull/214/files#diff-50f8bac543f30a639b3cf46d4f415b2bR105
Using the tests moved to pulp_file with https://pulp.plan.io/issues/5458 first 1) reproduce the issue and 2) determine where the time is being spent with cprofile capturing.
Related issues
Updated by bmbouter over 4 years ago
- Blocked by Task #5458: Port the performance-tests to pulp_file added
Updated by dalley over 4 years ago
I'm going to plug a couple of other tools that should be evaluated by the person who does this work. It might make your life easier.
Py-Spy
https://github.com/benfred/py-spy
This is sort of like cProfile, except that instead of tracing and recording the execution of every operation, it takes snapshots of the interpreter X times per second to see what functions are running for the most time. Unlike cProfile it does not influence the runtime performance of the operation -- running cProfile usually makes everything take several times longer than it otherwise would have which is painful if you have to re-run things frequently. It also has a nice live ncurses interface that you can use to sort and filter on certain parameters. And it can generate flamegraphs which are not quite as readable as cProfile reports but are servicable. You don't have to set it up beforehand, you can just point it at an already running PID.
pghero
https://github.com/ankane/pghero
If it turns out that the performance issues are in the database queries being made then pghero can help figure out why the queries are so inefficient. There's a nice live demo link here https://pghero.dokkuapp.com/datakick
pgadmin
https://www.pgadmin.org/
Big brother of pghero
.