Annif is an automated subject indexing toolkit. It was originally created as a statistical automated indexing tool that used metadata from the Finna.fi discovery interface as a training corpus.
This repo contains a rewritten production version of Annif based on the prototype.
Finto AI is a service based on Annif; see the source code of Finto AI and the 🤗 Hugging Face Hub collection containing the models Finto AI uses.
Annif is developed and tested on Linux. If you want to run Annif on Windows or Mac OS, the recommended way is to use Docker (see below) or a Linux virtual machine.
You will need Python 3.9-3.12 to install Annif.
The recommended way is to install Annif from PyPI into a virtual environment.
python3 -m venv annif-venv
source annif-venv/bin/activate
pip install annif
Start up the application:
annif
See Getting Started in the wiki for more details.
Annif supports tab-key completion in bash, zsh and fish shells for commands and options and project id, vocabulary id and path parameters.
To enable the completion support in your current terminal session use annif completion
command with the option according to your shell to produce the completion script and
source it. For example, run
source <(annif completion --bash)
To enable the completion support in all new sessions first add the completion script in your home directory:
annif completion --bash > ~/.annif-complete.bash
Then make the script to be automatically sourced for new terminal sessions by adding the
following to your ~/.bashrc file (or in some alternative startup
file):
source ~/.annif-complete.bash
For details and usage for other shells see Click documentation.
You can use Annif as a pre-built Docker container image from quay.io/natlibfi/annif repository. Please see the wiki documentation for details.
A development version of Annif can be installed by cloning the GitHub repository. Poetry is used for managing dependencies and virtual environment for the development version.
See CONTRIBUTING.md for information on unit tests, code style, development flow etc. details that are useful when participating in Annif development.
Clone the repository.
Switch into the repository directory.
Install pipx and Poetry if you don't have them. First pipx:
python3 -m pip install --user pipx
python3 -m pipx ensurepath
Open a new shell, and then install Poetry:
pipx install poetry
Poetry can be installed also without pipx: check the Poetry documentation.
Create a virtual environment and install dependencies:
poetry install
By default development dependencies are included. Use option -E to install dependencies for selected optional features (-E "extra1 extra2" for multiple extras), or install all of them with --all-extras. By default the virtual environment directory is not under the project directory, but there is a setting for selecting this.
Enter the virtual environment:
poetry shell
Start up the application:
annif
Annif can be tried out in the GitHub Codespaces. Just open a page for configuring a new codespace via the badge below, start the codespace from the green "Create codespace" button, and a terminal session will start in your browser with the contents of the Annif-tutorial repository:
Many resources are available:
See below for some articles about Annif in peer-reviewed Open Access journals. The software itself is also archived on Zenodo and has a citable DOI.
See "Cite this repository" in the details of the repository.
@article{golub2024annif,
title={Automated Dewey Decimal Classification of Swedish library metadata using Annif software},
author={Golub, Koraljka and Suominen, Osma and Mohammed, Ahmed Taiye and Aagaard, Harriet and Osterman, Olof},
journal={J. Doc.},
year={in press},
doi = {10.1108/JD-01-2022-0026},
url={https://www.emerald.com/insight/content/doi/10.1108/JD-01-2022-0026},
}
@article{suominen2022annif,
title={Annif and Finto AI: Developing and Implementing Automated Subject Indexing},
author={Suominen, Osma and Inkinen, Juho and Lehtinen, Mona},
journal={JLIS.it},
volume={13},
number={1},
pages={265--282},
year={2022},
doi = {10.4403/jlis.it-12740},
url={https://www.jlis.it/index.php/jlis/article/view/437},
}
@article{suominen2022analyzer,
title={Annif Analyzer Shootout: Comparing text lemmatization methods for automated subject indexing},
author={Suominen, Osma and Koskenniemi, Ilkka},
journal={Code4Lib J.},
number={54},
year={2022},
url={https://journal.code4lib.org/articles/16719},
}
@article{suominen2019annif,
title={Annif: DIY automated subject indexing using multiple algorithms},
author={Suominen, Osma},
journal={{LIBER} Quarterly},
volume={29},
number={1},
pages={1--25},
year={2019},
doi = {10.18352/lq.10285},
url = {https://doi.org/10.18352/lq.10285}
}
The code in this repository is licensed under Apache License 2.0, except for the
dependencies included under annif/static/css and annif/static/js,
which have their own licenses, see the file headers for details.
Please note that the YAKE library is licended
under GPLv3, while Annif is
licensed under the Apache License 2.0. The licenses are compatible, but
depending on legal interpretation, the terms of the GPLv3 (for example the
requirement to publish corresponding source code when publishing an executable
application) may be considered to apply to the whole of Annif+Yake if you
decide to install the optional Yake dependency.
FAQs
Automated subject indexing and classification tool
We found that annif demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.