This tool is used to compile a TensorFlow Lite for Microcontrollers neural network model into an optimised version that can run on an embedded system containing an Arm Ethos-U NPU.
In order to be accelerated by the Ethos-U NPU the network operators must be quantised to either 8-bit (unsigned or signed) or 16-bit (signed).
The optimised model will contain TensorFlow Lite Custom operators for those parts of the model that can be accelerated by the Ethos-U NPU. Parts of the model that cannot be accelerated are left unchanged and will instead run on the Cortex-M series CPU using an appropriate kernel (such as the Arm optimised CMSIS-NN kernels).
After compilation the optimised model can only be run on an Ethos-U NPU embedded system.
The tool will also generate performance estimates (EXPERIMENTAL) for the compiled model.
The tool has limited functionality for compiling a TOSA neural network (EXPERIMENTAL).
Vela is tested by comparing the bit exact numerical behaviour of the Ethos-U optimised operators against that of the corresponding TensorFlow Lite reference kernels (or TensorFlow Lite for Microcontrollers reference kernels in the case of the UNIDIRECTIONAL_SEQUENCE_LSTM operator). The following list indicates which version is used for comparison:
The majority of Vela's testing is done using a single version of Python, as indicated by the first version in the list below. However, some additional testing is also performed across a range of newer versions starting at the minimum version (pyproject.toml:project.requires-python) indicated in the brackets:
Vela runs on Linux and Microsoft Windows 10 operating systems.
The following should be installed prior to the installation of Vela:
apt install python3.10-dev or yum install python310-develVela is available to install as binary wheels or a source distribution from PyPi, or as source code from ML Platform. Both methods will automatically install all the required dependencies.
Install Vela from PyPi using the following command:
pip3 install ethos-u-vela
First obtain the source code by either downloading the desired TGZ file from:
https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ethos-u-vela
Or by cloning the git repository:
git clone https://review.mlplatform.org/ml/ethos-u/ethos-u-vela.git
Once you have the source code, Vela can be installed using the following command from the root directory of the repository:
pip3 install .
If you plan to modify the Vela codebase then it is recommended to install Vela
as an editable package to avoid the need to re-install after every modification.
This is done by adding the -e option to the install command like so:
pip3 install -e .[dev]
If you plan to contribute to the Vela project (highly encouraged!) then it is recommended to install Vela with the development dependencies (see Vela Testing for more details).
The C++ part of the the Vela compiler can be configured through the following environment variables:
| Variable | Description |
|---|---|
| CMAKE_BUILD_TYPE | Control cmake-build-type (Release or Debug) |
| CMAKE_BUILD_PARALLEL_LEVEL | Control parallel build level |
| CMAKE_GENERATOR | Override the default CMAKE generator |
| CMAKE_ARGS | Provide additional build-time options (see table below) |
The following build-time options can be provided through CMAKE_ARGS
# Example (Linux Bash)
CMAKE_ARGS="-DREGOR_ENABLE_LTO=OFF -DREGOR_ENABLE_WERROR=ON" pip3 install -e .[dev]
| Option | Description | Arguments |
|---|---|---|
| REGOR_ENABLE_LTO | Enable Link Time Optimization | ON/OFF |
| REGOR_ENABLE_LDGOLD | Enable Gold linker if available | ON/OFF |
| REGOR_ENABLE_CCACHE | Enable ccache if available | ON/OFF |
| REGOR_ENABLE_WERROR | Enable warnings as errors | ON/OFF |
| REGOR_ENABLE_STD_STATIC | Link libstdc and libgcc statically | ON/OFF |
| REGOR_ENABLE_COVERAGE | Enable Coverage build | ON/OFF |
| REGOR_ENABLE_PROFILING | Enable timer based runtime profiling | ON/OFF |
| REGOR_ENABLE_ASSERT | Enable asserts | ON/OFF |
| REGOR_ENABLE_EXPENSIVE_CHECKS | Enable expensive STL GLICXX asserts | ON/OFF |
| REGOR_ENABLE_RTTI | Enable RTTI (run-time type information) | ON/OFF |
| REGOR_ENABLE_VALGRIND | Enable Valgrind during check target | ON/OFF |
| REGOR_ENABLE_TESTING | Enable unit testing | ON/OFF |
| REGOR_ENABLE_CPPCHECK | Enable CPPCHECK | ON/OFF |
| REGOR_SANITIZE | Sanitizer setting (forwards to fsanitize) | String |
| REGOR_LOG_TRACE_MASK | Log trace enable mask | int (0->7) (See common/logging.hpp) |
| REGOR_PACKAGE_NAME | CPack package name | String |
| REGOR_DEBUG_COMPRESSION | Debug symbol compression | none, zlib, zlib-gnu |
| REGOR_PYTHON_BINDINGS_DESTINATION | Python bindings install destination | String |
| REGOR_PYEXT_VERSION | Python extension version | String |
Vela is run with an input .tflite or .tosa (EXPERIMENTAL) file passed on the
command line. This file contains the neural network to be compiled. The tool then
outputs an optimised .tflite file with a _vela suffix in the file name, along
with performance estimate (EXPERIMENTAL) CSV files, all to the output directory.
It also prints a performance estimation summary back to the console, see
Vela Performance Estimation Summary.
Example usage:
my_model.tflite. The optimised version will be output
to ./output/my_network_vela.tflite.vela my_model.tflite
/path/to/my_model.tflite and specify the output to go
in the directory ./results_dir/.vela --output-dir ./results_dir /path/to/my_model.tflite
vela --accelerator-config ethos-u65-512 my_model.tflite
vela --optimise Size my_model.tflite
vela --optimise Performance my_model.tflite
vela --optimise Performance --arena-cache-size 300000 my_model.tflite
My_Sys_Config
system configuration along with the My_Mem_Mode memory mode from the vela.ini
configuration file located in the config_files directory.vela --config Arm/vela.ini --system-config My_Sys_Config --memory-mode My_Mem_Mode my_model.tflite
vela --list-config-files
vela --help
When running the Vela compiler it may report a number of warning messages to the console. These should all be thoroughly reviewed as they will indicate decisions that the compiler has made in order to create the optimised network.
Some example networks that contain quantised operators which can be compiled by Vela to run on the Ethos-U NPU can be found at: https://tfhub.dev/s?deployment-format=lite&q=quantized
Once ethos-u-vela is installed, the user might want to install a different NumPy version that is still within the dependency constraints defined in pyproject.toml.
In some scenarios, doing so might prevent ethos-u-vela from functioning as expected due to incompatibilities between the installed NumPy C headers used in the mlw_codec and the current version of NumPy.
Example scenario:
In the ethos-u-vela source directory, run:
virtualenv -p 3.10 venv
. venv/bin/activate
pip install ethos-u-vela
Next, install a different NumPy version (e.g. 1.21.3)
pip install numpy==1.21.3 --force
Finally, run ethos-u-vela. You might get an error similar to this:
ImportError: NumPy C API version mismatch
(Build-time version: 0x10, Run-time version: 0xe)
This is a known issue most likely caused by a change in the API version in
NumPy after installing ethos-u-vela.
In order for ethos-u-vela to work with an older version of NumPy that uses different C APIs, you will need to install the desired NumPy version first, and then build ethos-u-vela with that specific NumPy version:
Uninstall ethos-u-vela and install the desired version of NumPy
pip uninstall ethos-u-vela
pip install numpy==1.21.3 --force
Install required build dependencies
pip install "setuptools_scm[toml]<6" wheel
Install ethos-u-vela without build isolation. Not using build isolation ensures that the correct version of NumPy is used when copying the C headers in mlw_codec during the build process.
pip install ethos-u-vela --no-build-isolation --no-cache-dir
Please see Vela External APIs.
Please see Vela Community Bug Reporting for a description of how to report bugs.
Please see Vela Contributions.
Please see Vela Debug Database.
This product conforms to Arm’s inclusive language policy and, to the best of our knowledge, does not contain any non-inclusive language. If you find something that concerns you, email terms@arm.com.
Please see Vela CLI Options. This includes a description of the system configuration file format.
Please see Vela Performance Estimation Summary. NOTE: This is only an estimate. For performance numbers we recommend running the compiled network on an FVP Model or FPGA.
Please see Vela Releases.
Additional useful information:
Please see Vela Security.
Please see Vela Supported Operators for the list of operators supported in this release.
Please see Vela Testing.
Vela is licensed under Apache License 2.0.
FAQs
Neural network model compiler for Arm Ethos-U NPUs
We found that ethos-u-vela demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 3 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Malicious Go packages are impersonating popular libraries to install hidden loader malware on Linux and macOS, targeting developers with obfuscated payloads.
Security News
Bybit's $1.46B hack by North Korea's Lazarus Group pushes 2025 crypto losses to $1.6B in just two months, already surpassing all of 2024's $1.49B total.
Security News
OpenSSF has published OSPS Baseline, an initiative designed to establish a minimum set of security-related best practices for open source software projects.