IMPORTANT NOTE
This is the legacy adapter-transformers
library, which has been replaced by the new Adapters library, found here: https://github.com/adapter-hub/adapters.
Install the new library via pip: pip install adapters
.
This repository is kept for archival purposes, and will not be updated in the future.
Please use the new library for all active projects.
The documentation of this library can be found at https://docs-legacy.adapterhub.ml.
The documentation of the new Adapters library can be found at https://docs.adapterhub.ml.
For transitioning, please read: https://docs.adapterhub.ml/transitioning.html.
adapter-transformers
A friendly fork of HuggingFace's Transformers, adding Adapters to PyTorch language models
adapter-transformers
is an extension of HuggingFace's Transformers library, integrating adapters into state-of-the-art language models by incorporating AdapterHub, a central repository for pre-trained adapter modules.
💡 Important: This library can be used as a drop-in replacement for HuggingFace Transformers and regularly synchronizes new upstream changes.
Thus, most files in this repository are direct copies from the HuggingFace Transformers source, modified only with changes required for the adapter implementations.
Installation
adapter-transformers
currently supports Python 3.8+ and PyTorch 1.12.1+.
After installing PyTorch, you can install adapter-transformers
from PyPI ...
pip install -U adapter-transformers
... or from source by cloning the repository:
git clone https://github.com/adapter-hub/adapter-transformers.git
cd adapter-transformers
pip install .
Getting Started
HuggingFace's great documentation on getting started with Transformers can be found here. adapter-transformers
is fully compatible with Transformers.
To get started with adapters, refer to these locations:
Implemented Methods
Currently, adapter-transformers integrates all architectures and methods listed below:
Supported Models
We currently support the PyTorch versions of all models listed on the Model Overview page in our documentation.
Citation
If you use this library for your work, please consider citing our paper AdapterHub: A Framework for Adapting Transformers:
@inproceedings{pfeiffer2020AdapterHub,
title={AdapterHub: A Framework for Adapting Transformers},
author={Pfeiffer, Jonas and
R{\"u}ckl{\'e}, Andreas and
Poth, Clifton and
Kamath, Aishwarya and
Vuli{\'c}, Ivan and
Ruder, Sebastian and
Cho, Kyunghyun and
Gurevych, Iryna},
booktitle={Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations},
pages={46--54},
year={2020}
}