vame-py

🌟 Welcome to EthoML/VAME (Variational Animal Motion Encoding), an open-source machine learning tool for behavioral action segmentation and analyses.

VAME documentation.

❗ Clear here to read the NEW peer-reviewed and open-access neuroscience article in Cell Reports. ❗

We are a group of behavioral enthusiasts, comprising the original VAME developers Kevin Luxem and Pavol Bauer, behavioral neuroscientists Stephanie R. Miller and Jorge J. Palop, and computer scientists and statisticians Alex Pico, Reuben Thomas, and Katie Ly. Our aim is to provide scalable, unbiased and sensitive approaches for assessing mouse behavior using computer vision and machine learning approaches.

We are focused on the expanding the analytical capabilities of VAME segmentation by providing curated scripts for VAME implementation and tools for data processing, visualization, and statistical analyses.

Recent Improvements to VAME

• Curated scripts for VAME implementation
• Addition of compatability with DeepLabCut, SLEAP, and LightningPose
• Addition of compatability with movement for data ingestion
• Addition of a new cost function for community dendrogram generation
• Addition of a new egocentric alignment method
• Refined output filename structure

Authors and Code Contributors

VAME was developed by Kevin Luxem and Pavol Bauer (Luxem et. al., 2022). The original VAME repository was deprecated, forked, and is now being maintained here at https://github.com/EthoML/VAME.

The development of VAME is heavily inspired by DeepLabCut. As such, the VAME project management codebase has been adapted from the DeepLabCut codebase. The DeepLabCut 2.0 toolbox is © A. & M.W. Mathis Labs deeplabcut.org, released under LGPL v3.0. The implementation of the VRAE model is partially adapted from the Timeseries clustering repository developed by Tejas Lodaya.

VAME in a Nutshell

VAME is a framework to cluster behavioral signals obtained from pose-estimation tools. It is a PyTorch-based deep learning framework which leverages the power of recurrent neural networks (RNN) to model sequential data. In order to learn the underlying complex data distribution, we use the RNN in a variational autoencoder setting to extract the latent state of the animal in every step of the input time series. The workflow of VAME consists of 5 steps and we explain them in detail here

Installation

To get started we recommend using Anaconda with Python 3.11 or higher. Here, you can create a virtual enviroment to store all the dependencies necessary for VAME. You can also use the environment-<os>.yaml files supplied here, by simply opening the terminal, running git clone https://github.com/LINCellularNeuroscience/VAME.git, then typ cd VAME then run: conda env create -f environment-<os>.yaml).

• Go to the locally cloned VAME directory and run python setup.py install in order to install VAME in your active conda environment.
• Install the current stable Pytorch release using the OS-dependent instructions from the Pytorch website. Currently, VAME is tested on PyTorch 2.2.2. (Note, if you use the conda file we supply, PyTorch is already installed and you don't need to do this step.)

Getting Started

First, you should make sure that you have a GPU powerful enough to train deep learning networks. In our original 2022 paper, we were using a single Nvidia GTX 1080 Ti GPU to train our network. A hardware guide can be found here. VAME can also be trained in Google Colab or on a HPC cluster. Once you have your computing setup ready, begin using VAME by following the workflow guide.

Once you have VAME installed, you can try VAME out on a set of mouse behavioral videos and .csv files publicly available in the examples folder.

References

New 2024 Miller et al.: Machine learning reveals prominent spontaneous behavioral changes and treatment efficacy in humanized and transgenic Alzheimer's disease models
Original 2022 Luxem et al.: Identifying Behavioral Structure from Deep Variational Embeddings of Animal Motion
See also:
Mocellin et al.: A septal-ventral tegmental area circuit drives exploratory behavior
Kingma & Welling: Auto-Encoding Variational Bayes
Pereira & Silveira: Learning Representations from Healthcare Time Series Data for Unsupervised Anomaly Detection

License: GPLv3

See the LICENSE file for the full statement.

Code Reference (DOI)