VizTracer
VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.
The front-end UI is powered by Perfetto. Use "AWSD" to zoom/navigate.
More help can be found in "Support - Controls".
Highlights
- Detailed function entry/exit information on timeline with source code
- Super easy to use, no source code change for most features, no package dependency
- Supports threading, multiprocessing, subprocess and async
- Powerful front-end, able to render GB-level trace smoothly
- Works on Linux/MacOS/Windows
Install
The preferred way to install VizTracer is via pip
pip install viztracer
Basic Usage
Command Line
Assume you have a python script to run:
python3 my_script.py arg1 arg2
You can simply use VizTracer by
viztracer my_script.py arg1 arg2
A result.json
file will be generated, which you can open with vizviewer
vizviewer will host an HTTP server on http://localhost:9001
. You can also open your browser and use that address.
If you do not want vizviewer to open the webbrowser automatically, you can use
vizviewer --server_only result.json
If you just need to bring up the trace report once, and do not want the persistent server, use
vizviewer --once result.json
vizviewer result.json
vizviewer ./
vizviewer --use_external_processor result.json
A VS Code Extension
is available to make your life even easier.
Add --open
to open the reports right after tracing
viztracer --open my_script.py arg1 arg2
viztracer -o result.html --open my_script.py arg1 arg2
modules and console scripts(like flask
) are supported as well
viztracer -m your_module
viztracer flask run
Inline
You can also manually start/stop VizTracer in your script as well.
from viztracer import VizTracer
tracer = VizTracer()
tracer.start()
tracer.stop()
tracer.save()
Or, you can do it with with
statement
with VizTracer(output_file="optional.json") as tracer:
Jupyter
If you are using Jupyter, you can use viztracer cell magics.
%load_ext viztracer
%%viztracer
A VizTracer Report
button will appear after the cell and you can click it to view the results
Advanced Usage
Trace Filter
VizTracer can filter out the data you don't want to reduce overhead and keep info of a longer time period before you dump the log.
VizTracer can log extra information without changing your source code
Add Custom Event
VizTracer supports inserting custom events while the program is running. This works like a print debug, but you can know when this print happens while looking at trace data.
Misc
Multi Thread Support
VizTracer supports python native threading
module without the need to do any modification to your code. Just start VizTracer
before you create threads and it will just work.
For other multi-thread scenarios, you can use enable_thread_tracing()
to notice VizTracer about the thread to trace it.
Refer to multi thread docs for details
Multi Process Support
VizTracer supports subprocess
, multiprocessing
, os.fork()
, concurrent.futures
, and loky
out of the box.
For more general multi-process cases, VizTracer can support with some extra steps.
Refer to multi process docs for details
Async Support
VizTracer supports asyncio
natively, but could enhance the report by using --log_async
.
Refer to async docs for details
Flamegraph
VizTracer can show flamegraph of traced data.
vizviewer --flamegraph result.json
Remote attach
VizTracer supports remote attach to an arbitrary Python process to trace it, as long as viztracer is importable
Refer to remote attach docs
JSON alternative
VizTracer needs to dump the internal data to json format. It is recommended for the users to install orjson
, which is much faster than the builtin json
library. VizTracer will try to import orjson
and fall back to the builtin json
library if orjson
does not exist.
Performance
VizTracer will introduce 2x to 3x overhead in the worst case. The overhead is much better if there are less function calls or if filters are applied correctly.
An example run for test_performance with Python 3.8 / Ubuntu 18.04.4 on Github VM
fib:
0.000678067(1.00)[origin]
0.019880272(29.32)[py] 0.011103901(16.38)[parse] 0.021165599(31.21)[json]
0.001344933(1.98)[c] 0.008181911(12.07)[parse] 0.015789866(23.29)[json]
0.001472846(2.17)[cProfile]
hanoi (6148, 4100):
0.000550255(1.00)[origin]
0.016343521(29.70)[py] 0.007299123(13.26)[parse] 0.016779364(30.49)[json]
0.001062505(1.93)[c] 0.006416136(11.66)[parse] 0.011463236(20.83)[json]
0.001144914(2.08)[cProfile]
qsort (8289, 5377):
0.002817679(1.00)[origin]
0.052747431(18.72)[py] 0.011339725(4.02)[parse] 0.023644345(8.39)[json]
0.004767673(1.69)[c] 0.008735166(3.10)[parse] 0.017173703(6.09)[json]
0.007248019(2.57)[cProfile]
slow_fib (1135, 758):
0.028759652(1.00)[origin]
0.033994071(1.18)[py] 0.001630461(0.06)[parse] 0.003386635(0.12)[json]
0.029481623(1.03)[c] 0.001152415(0.04)[parse] 0.002191417(0.08)[json]
0.028289305(0.98)[cProfile]
Documentation
For full documentation, please see https://viztracer.readthedocs.io/en/stable
Bugs/Requests
Please send bug reports and feature requests through github issue tracker. VizTracer is currently under development now and it's open to any constructive suggestions.
License
Copyright 2020-2023 Tian Gao.
Distributed under the terms of the Apache 2.0 license.