pyunormalize

pyunormalize

A pure-Python implementation of the Unicode normalization algorithm. This package conforms strictly to version 17.0 of the Unicode Standard, released in September 2025, by using its own dedicated data derived from the Unicode character database (UCD) corresponding to that version. This approach ensures total independence from the Unicode version built into the host Python environment.

All normalization functions (NFC, NFD, NFKC, and NFKD) have been thoroughly tested against the official Unicode test file for accuracy.

Python support policy

This package requires Python 3.8 or newer.

In version 17.0.0 of pyunormalize, support for Python 3.6 and 3.7 was dropped. These interpreters are past end-of-life and cannot be used reliably with modern Python packaging and installation tools.

Installation and updates

To install the package, run:

pip install pyunormalize

To upgrade to the latest version, run:

pip install pyunormalize --upgrade

Changelog

Check out the latest updates and changes here.

Unicode character database (UCD) version

Retrieve the version of the Unicode character database in use:

>>> from pyunormalize import UCD_VERSION
>>> print(UCD_VERSION)
17.0.0

Usage examples

Base normalization forms

This section demonstrates the use of the NFC, NFD, NFKC, and NFKD functions for specific normalization tasks.

from pyunormalize import NFC, NFD, NFKC, NFKD

# Example string with accented characters
text = "élève"  # "\u00E9\u006C\u00E8\u0076\u0065"

# Normalize to different forms
nfc = NFC(text)
nfd = NFD(text)

# Comparisons
print(nfc == text)  # True  (NFC preserves the original composed form)
print(nfd == nfc)   # False (NFD decomposes accented characters)

# Display Unicode code points in hexadecimal
print("NFC: " + " ".join([f"{ord(c):04X}" for c in nfc]))
print("NFD: " + " ".join([f"{ord(c):04X}" for c in nfd]))
# Output:
# NFC: 00E9 006C 00E8 0076 0065
# NFD: 0065 0301 006C 0065 0300 0076 0065

Using the generic normalize function

The normalize function can be used to apply any normalization form programmatically.

from pyunormalize import normalize

text = "désaffiliât"
print("Input\t" + " ".join([f"{ord(c):04X}" for c in text]))

# Apply each of the four forms
forms = ["NFC", "NFD", "NFKC", "NFKD"]
for form in forms:
    normalized_text = normalize(form, text)
    hex_repr = " ".join([f"{ord(c):04X}" for c in normalized_text])
    print(f"{form}\t{hex_repr}")

# Output:
# Input   0064 00E9 0073 0061 FB03 006C 0069 00E2 0074
# NFC     0064 00E9 0073 0061 FB03 006C 0069 00E2 0074
# NFD     0064 0065 0301 0073 0061 FB03 006C 0069 0061 0302 0074
# NFKC    0064 00E9 0073 0061 0066 0066 0069 006C 0069 00E2 0074
# NFKD    0064 0065 0301 0073 0061 0066 0066 0069 006C 0069 0061 0302 0074

Related resources

This implementation is based on the following resources:

• The Unicode Standard, Version 17.0 – Core Specification, Section 3.11: “Normalization Forms”
• Unicode Standard Annex #15: “Unicode Normalization Forms,” revision 57

Licenses

The code is available under the terms of the MIT license.

Usage of Unicode data files is governed by the UNICODE TERMS OF USE. Further specifications of rights and restrictions pertaining to the use of the Unicode data files and software can be found in the Unicode Data Files and Software License, a copy of which is included as UNICODE-LICENSE.