Composer is a PyTorch library that enables you to train neural networks faster, at lower cost, and to higher accuracy.
Supercharge your Model Training
Deep Learning Framework for Training at Scale
[Website] - [Getting Started] - [Docs] - [We're Hiring!]
Composer is an open-source deep learning training library by MosaicML. Built on top of PyTorch, the Composer library makes it easier to implement distributed training workflows on large-scale clusters.
We built Composer to be optimized for scalability and usability, integrating best practices for efficient, multi-node training. By abstracting away low-level complexities like parallelism techniques, distributed data loading, and memory optimization, you can focus on training modern ML models and running experiments without slowing down.
We recommend using Composer to speedup your experimentation workflow if you’re training neural networks of any size, including:
Composer is heavily used by the MosaicML research team to train state-of-the-art models like MPT, and we open-sourced this library to enable the ML community to do the same. This framework is used by organizations in both the tech industry and the academic sphere and is continually updated with new features, bug fixes, and stability improvements for production workloads.
We designed Composer from the ground up for modern deep learning workloads. Gone are the days of AlexNet and ResNet, when state-of-the-art models could be trained on a couple of desktop GPUs. Today, developing the latest and greatest deep learning models often requires cluster-scale hardware — but with Composer’s help, you’ll hardly notice the difference.
The heart of Composer is our Trainer abstraction: a highly optimized PyTorch training loop designed to allow both you and your model to iterate faster. Our trainer has simple ways for you to configure your parallelization scheme, data loaders, metrics, loggers, and more.
Whether you’re training on 1 GPU or 512 GPUs, 50MB or 10TB of data - Composer is built to keep your workflow simple.
Other high-level deep learning trainers provide simplicity at the cost of rigidity. When you want to add your own features, their abstractions get in your way. Composer, on the other hand, provides simple ways for you to customize our Trainer to your needs.
Fig. 1: Composer’s training loop has a series of events that occur at each stage in the training process. Callbacks are functions that users write to run at specific events. For example, our Learning Rate Monitor Callback logs the learning rate at every BATCH_END event.
Composer is built to automate away low-level pain points and headaches so you can focus on the important (and fun) parts of deep learning and iterate faster.
Time class.Integrate with the tools you know and love for experiment tracking and data streaming.
Composer is designed for users who are comfortable with Python and have basic familiarity with deep learning fundamentals and PyTorch.
Software requirements: A recent version of PyTorch.
Hardware requirements: System with CUDA-compatible GPUs (AMD + RoCM coming soon!). Composer can run on CPUs, but for full benefits, we recommend using it on hardware accelerators.
Composer can be installed with pip:
pip install mosaicml
To simplify the environment setup for Composer, we also provide a set of pre-built Docker images. We highly recommend you use our Docker images.
Here is a code snippet demonstrating our Trainer on the MNIST dataset.
import torch
import torch.nn as nn
import torch.nn.functional as F
from torchvision import datasets, transforms
from torch.utils.data import DataLoader
from composer import Trainer
from composer.models import ComposerClassifier
from composer.algorithms import LabelSmoothing, CutMix, ChannelsLast
class Model(nn.Module):
"""Toy convolutional neural network architecture in pytorch for MNIST."""
def __init__(self, num_classes: int = 10):
super().__init__()
self.num_classes = num_classes
self.conv1 = nn.Conv2d(1, 16, (3, 3), padding=0)
self.conv2 = nn.Conv2d(16, 32, (3, 3), padding=0)
self.bn = nn.BatchNorm2d(32)
self.fc1 = nn.Linear(32 * 16, 32)
self.fc2 = nn.Linear(32, num_classes)
def forward(self, x):
out = self.conv1(x)
out = F.relu(out)
out = self.conv2(out)
out = self.bn(out)
out = F.relu(out)
out = F.adaptive_avg_pool2d(out, (4, 4))
out = torch.flatten(out, 1, -1)
out = self.fc1(out)
out = F.relu(out)
return self.fc2(out)
transform = transforms.Compose([transforms.ToTensor()])
dataset = datasets.MNIST("data", train=True, download=True, transform=transform)
train_dataloader = DataLoader(dataset, batch_size=128)
trainer = Trainer(
model=ComposerClassifier(module=Model(), num_classes=10),
train_dataloader=train_dataloader,
max_duration="2ep",
algorithms=[
LabelSmoothing(smoothing=0.1),
CutMix(alpha=1.0),
ChannelsLast(),
],
)
trainer.fit()
Next, check out our Getting Started Colab for a walk-through of Composer’s main features. In this tutorial, we will cover the basics of the Composer Trainer:
Once you’ve completed the Quick Start, you can go through the below tutorials or our documentation to further familiarize yourself with Composer.
If you have any questions, please feel free to reach out to us on our Community Slack!
Here are some resources actively maintained by the Composer community to help you get started:
| Resource | Details |
|---|---|
| Training BERTs with Composer and 🤗 | A Colab Notebook showing how to train BERT models with Composer and 🤗! |
| Pretraining and Finetuning an LLM Tutorial | A tutorial from MosaicML’s LLM Foundry, using MosaicML Composer, StreamingDataset, and MCLI on training and evaluating LLMs. |
| Migrating from PyTorch Lightning | A tutorial is to illustrating a path from working in PyTorch Lightning to working in Composer. |
| Finetuning and Pretraining HuggingFace Models | Want to use Hugging Face models with Composer? No problem. Here, we’ll walk through using Composer to fine-tune a pretrained Hugging Face BERT model. |
| Building Speedup Methods | A Colab Notebook showing how to build new training modifications on top of Composer |
Composer can be used on its own, but for the smoothest experience we recommend using it in combination with other components of the MosaicML ecosystem:
Here are some projects and experiments that used Composer. Got something to add? Share in our Community Slack!
Composer is part of the broader Machine Learning community, and we welcome any contributions, pull requests, or issues!
To start contributing, see our Contributing page.
P.S.: We're hiring!
@misc{mosaicml2022composer,
author = {The Mosaic ML Team},
title = {composer},
year = {2021},
howpublished = {\url{https://github.com/mosaicml/composer/}},
}
FAQs
Composer is a PyTorch library that enables you to train neural networks faster, at lower cost, and to higher accuracy.
We found that mosaicml demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 12 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.