xDSL is a Python-native compiler framework built around SSA-based intermediate representations (IRs). Users of xDSL build a compiler by assembling predefined domain-specific IRs and, optionally, defining their own custom IRs. xDSL uses multi-level IRs, meaning that during the compilation process, a program will be lowered through several of these IRs. This allows the implementation of abstraction-specific optimization passes, similar to the structure of common DSL compilers (such as Devito, Psyclone, and Firedrake). To simplify the writing of these passes, xDSL uses a uniform data structure based on SSA, basic blocks, and regions, which additionally enables the writing of generic passes.
The design of xDSL is influenced by MLIR, a compiler framework developed in C++, that is part of the LLVM project. An inherent advantage of a close design is the easy interaction between the two frameworks, making it possible to translate abstractions and programs back and forth. This results in one big SSA-based abstraction ecosystem that can be worked with through Python, making analysis through simple scripting languages possible. Additionally, xDSL can leverage MLIR's code generation and low-level optimization capabilities.
To use xDSL as part of a larger project for developing your own compiler, just install xDSL via pip:
pip install xdsl
Note: This version of xDSL is validated against a specific MLIR version,
interoperability with other versions may result in problems. The supported
MLIR version is commit d401987fe349a87c53fe25829215b080b70c0c1a.
xDSL has a number of subprojects, some of which require extra dependencies. In order to keep the set of dependencies ot a minimum, these extra dependencies have to be specified explicitly. To install these, use:
pip install xdsl[gui,jax,riscv,wgpu,onnx]
To install the testing/development dependencies, use:
pip install xdsl[dev]
These may be useful for projects wanting to replicate the xDSL testing setup.
Check out the dedicated Getting Started guide for a comprehensive tutorial.
To get familiar with xDSL, we recommend starting with our Jupyter notebooks. The notebooks consist of examples and documentation concerning the core xDSL data structures and the xDSL's Python-embedded abstraction definition language, as well as examples of implementing custom compilers, like a database compiler. There also exists a small documentation showing how to connect xDSL with MLIR for users interested in that use case.
We provide a Makefile containing a lot of common tasks, which might provide an overview of common actions.
To contribute to the development of xDSL follow the subsequent steps.
git clone https://github.com/xdslproject/xdsl.git
cd xdsl
# set up the venv and install everything
make venv
# activate the venv
source venv/bin/activate
The xDSL project uses pytest unit tests and LLVM-style filecheck tests. They can be executed from the root directory:
# Executes pytests which are located in tests/
pytest
# Executes filecheck tests
lit tests/filecheck
# run all tests using makefile
make tests
All python code used in xDSL uses ruff to format the code in a uniform manner.
To automate the formatting, we use pre-commit hooks from the pre-commit package.
# Install the pre-commit on your `.git` folder
make precommit-install
# to run the hooks:
make precommit
# alternatively, run ruff directly:
ruff format
Furthermore, all python code must run through pyright
without errors. Pyright can be run on all staged files through the
makefile using make pyright.
[!IMPORTANT]
Experimental Pyright Features
xDSL currently relies on an experimental feature of Pyright called TypeForm TypeForm is in discussion and will likely land in some future version of Python.
For xDSL to type check correctly using Pyright, please add this to your
pyproject.toml:[tool.pyright] enableExperimentalFeatures = true
You can also join the discussion at our Zulip chat room, kindly supported by community hosting from Zulip.
FAQs
xDSL
We found that xdsl demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 6 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
PyPI confirms no security flaws were exploited in the Ultralytics supply chain attack and highlights improvements for safer package publishing.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.