pynini

OpenGrm Pynini

This is a a Python extension module for compiling, optimizing and applying grammar rules. Rules can be compiled into weighted finite state transducers, pushdown transducers, or multi-pushdown transducers. It uses OpenFst finite-state transducers (FSTs) and FST archives (FArs) as inputs and outputs.

This library is primarily developed by Kyle Gorman.

If you use Pynini in your research, we would appreciate if you cite the following paper:

K. Gorman. 2016. Pynini: A Python library for weighted finite-state grammar compilation. In Proc. ACL Workshop on Statistical NLP and Weighted Automata, 75-80.

(Note that some of the code samples in the paper are now out of date and not expected to work.)

Dependencies

• A standards-compliant C++17 compiler (GCC >= 7 or Clang >= 700)
• The compatible recent version of OpenFst (see NEWS for this) built with the grm extensions (i.e., built with ./configure --enable-grm) and headers
• Python 3.6+ and headers

Installation instructions

There are various ways to install Pynini depending on your platform.

Windows

While Pynini is neither designed for nor tested on Windows, it can be installed using the Windows Subsystem for Linux (WSL). Simply enter the WSL environment and follow the Linux instructions below.

MacOS

The pre-compiled library can be installed from conda-forge by running conda install -c conda-forge pynini.

Alternatively, one can build from source from PyPI by running pip install pynini.

Finally, one can use Bazel to build from source by running bazel build //:all anywhere in the source tree.

Linux

The pre-compiled library can be installed from conda-forge by running conda install -c conda-forge pynini.

Alternatively, one can install a pre-compiled manylinux wheel from PyPI by running pip install pynini. This will install the pre-compiled manylinux wheel (if available for the release and compatible with your platform), and build and install from source if not. Unlike the conda-forge option above, which also installs OpenFst and Graphviz, this does not install the OpenFst or Graphviz command-line tools. See the enclosed Dockerfile for instructions for building and deploying manylinux wheels.

Finally, one can use Bazel to build from source by running bazel build //:all anywhere in the source tree.

Testing

To confirm successful installation, run pip install -r requirements, then python tests/pynini_test.py. If all tests pass, the final line will read OK; a successful run will log some errors to STDERR (this is working as expected).

Python version support

Pynini 2.0.0 and onward support Python 3. Pynini 2.1 versions (onward) drop Python 2 support. The current release supports Python 3.8--3.13.

License

Pynini is released under the Apache license. See LICENSE for more information.

Interested in contributing?

See CONTRIBUTING for more information.

Mandatory disclaimer

This is not an official Google product.