Print the stack trace of a running Python process, or of a Python core dump.
PyStack is a tool that uses forbidden magic to let you inspect the stack frames of a running Python process or a Python core dump, helping you quickly and easily learn what it's doing (or what it was doing when it crashed) without having to interpret nasty CPython internals.
PyStack has the following amazing features:
At this time only Linux is supported.
If you wish to build PyStack from source, you need the following binary dependencies in your system:
Note that sometimes both libraries are provided together as part of a distribution's elfutils
package.
Check your package manager on how to install these dependencies (e.g.,
apt-get install libdw-dev libelf-dev in Debian-based systems). Note that you may need to tell the
compiler where to find the header and library files of the dependencies for the build to succeed.
If pkg-config is available (e.g. apt-get install pkg-config on Debian-based systems), it will
automatically be used to locate the libraries and configure the correct build flags.
Check your distribution's documentation to determine the location of the header and library files
or for more detailed information. When building on Alpine Linux (or any other distribution that
doesn't use glibc) you'll need elfutils 0.188 or newer. You may need to build this from source if
your distribution's package manager doesn't have it.
Once you have these binary dependencies installed, you can clone the repository and follow the typical build process for Python libraries:
git clone git@github.com:bloomberg/pystack.git pystack
cd pystack
python3 -m venv ../pystack-env/ # just an example, put this wherever you want
source ../pystack-env/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install -e .
python3 -m pip install -r requirements-test.txt -r requirements-extra.txt
This will install PyStack in the virtual environment in development mode (the -e of the last
pip install command), and then install the Python libraries needed to test it, lint it, and
generate its documentation.
If you plan to contribute back, you should install the pre-commit hooks:
pre-commit install
This will ensure that your contribution passes our linting checks.
You can find the full documentation here.
PyStack uses distinct subcommands for analyzing running processes and core dump files.
usage: pystack [-h] [-v] [--no-color] {remote,core} ...
Get Python stack trace of a remote process
options:
-h, --help show this help message and exit
-v, --verbose
--no-color Deactivate colored output
commands:
{remote,core} What should be analyzed by PyStack (use <command> --help for a command-specific help section).
remote Analyze a remote process given its PID
core Analyze a core dump file given its location and the executable
The remote command is used to analyze the status of a running (remote) process. The analysis is
always done in a safe and non-intrusive way, as no code is loaded in the memory space of the
process under analysis and no memory is modified in the remote process. This makes analysis using
PyStack a great option even for those services and applications that are running in environments
where the running process must not be impacted in any way (other than being temporarily paused,
though --no-block can avoid even that). There are several options available:
usage: pystack remote [-h] [-v] [--no-color] [--no-block] [--native] [--native-all] [--locals] [--exhaustive] pid
positional arguments:
pid The PID of the remote process
options:
-h, --help show this help message and exit
-v, --verbose
--no-color Deactivate colored output
--no-block do not block the process when inspecting its memory
--native Include the native (C) frames in the resulting stack trace
--native-all Include native (C) frames from threads not registered with the interpreter (implies --native)
--locals Show local variables for each frame in the stack trace
--exhaustive Use all possible methods to obtain the Python stack info (may be slow)
To use PyStack, you just need to provide the PID of the process:
$ pystack remote 112
Traceback for thread 112 [] (most recent call last):
(Python) File "/test.py", line 17, in <module>
first_func()
(Python) File "/test.py", line 6, in first_func
second_func()
(Python) File "/test.py", line 10, in second_func
third_func()
(Python) File "/test.py", line 14, in third_func
time.sleep(1000)
The core subcommand is used to analyze the status of a core dump file. Analyzing core files is
very similar to analyzing processes but there are some differences, as the core file does not
contain the totality of the memory that was valid when the program was live. In most cases, this
makes no difference, as PyStack will try to adapt automatically. However, in some cases, you will
need to specify extra command line options to help PyStack locate the information it needs. When
analyzing cores, there are several options available:
usage: pystack core [-h] [-v] [--no-color] [--native] [--native-all] [--locals] [--exhaustive] [--lib-search-path LIB_SEARCH_PATH | --lib-search-root LIB_SEARCH_ROOT] core [executable]
positional arguments:
core The path to the core file
executable (Optional) The path to the executable of the core file
options:
-h, --help show this help message and exit
-v, --verbose
--no-color Deactivate colored output
--native Include the native (C) frames in the resulting stack trace
--native-all Include native (C) frames from threads not registered with the interpreter (implies --native)
--locals Show local variables for each frame in the stack trace
--exhaustive Use all possible methods to obtain the Python stack info (may be slow)
--lib-search-path LIB_SEARCH_PATH
List of paths to search for shared libraries loaded in the core. Paths must be separated by the ':' character
--lib-search-root LIB_SEARCH_ROOT
Root directory to search recursively for shared libraries loaded into the core.
In most cases, you just need to provide the location of the core to use PyStack with core dump files:
$ pystack core ./the_core_file
Using executable found in the core file: /usr/bin/python3.8
Core file information:
state: t zombie: True niceness: 0
pid: 570 ppid: 1 sid: 1
uid: 0 gid: 0 pgrp: 570
executable: python3.8 arguments: python3.8
The process died due receiving signal SIGSTOP
Traceback for thread 570 [] (most recent call last):
(Python) File "/test.py", line 19, in <module>
first_func({1: None}, [1,2,3])
(Python) File "/test.py", line 7, in first_func
second_func(x, y)
(Python) File "/test.py", line 12, in second_func
third_func(x, y)
(Python) File "/test.py", line 16, in third_func
time.sleep(1000)
PyStack is Apache-2.0 licensed, as found in the LICENSE file.
This project has adopted a Code of Conduct. If you have any concerns about the Code, or behavior that you have experienced in the project, please contact us at opensource@bloomberg.net.
If you believe you have identified a security vulnerability in this project, please send an email to the project team at opensource@bloomberg.net, detailing the suspected issue and any methods you've found to reproduce it.
Please do NOT open an issue in the GitHub repository, as we'd prefer to keep vulnerability reports private until we've had an opportunity to review and address them.
We welcome your contributions to help us improve and extend this project!
Below you will find some basic steps required to be able to contribute to the project. If you have any questions about this process or any other aspect of contributing to a Bloomberg open source project, feel free to send an email to opensource@bloomberg.net and we'll get your questions answered as quickly as we can.
Since this project is distributed under the terms of an open source license, contributions that you make are licensed under the same terms. For us to be able to accept your contributions, we will need explicit confirmation from you that you are able and willing to provide them under these terms, and the mechanism we use to do this is called a Developer's Certificate of Origin (DCO). This is similar to the process used by the Linux kernel, Samba, and many other major open source projects.
To participate under these terms, all that you must do is include a line like the following as the last line of the commit message for each commit in your contribution:
Signed-Off-By: Random J. Developer <random@developer.example.org>
The simplest way to accomplish this is to add -s or --signoff to your git commit command.
You must use your real name (sorry, no pseudonyms, and no anonymous contributions).
FAQs
Analysis of the stack of remote python processes
We found that pystack demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 3 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
PyPI confirms no security flaws were exploited in the Ultralytics supply chain attack and highlights improvements for safer package publishing.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.