pabo

What is this?

Parsing binary data from Python has always been a bit of a pain, thanks to the weirdly designed struct module in Python's standard library. struct uses format strings to specify the layout of binary data, where each character specifies the type of data being packed/unpacked. But no can remember the format characters to begin with! This has led to numerous packages cropping in an attempt to solve the problem, such as:

• bread
• construct
• structures

and many others. pabo is my response to such packages. It makes parsing binary data so easy, anyone could do it! For example, here is how you would parse the beginning of a PNG file to get the width and height of the image:

import pabo as pb

png = pb.Spec(
    {
        "magic": pb.Const(
            b"\x89PNG\x0d\x0a\x1a\x0a",
            pb.Bytes(8),
        ),
        "ihdr_size": pb.Int(4, endian="big"),
        "ihdr_id": pb.Const(b"IHDR", pb.Bytes(4)),
        "width": pb.Int(4, endian="big"),
        "height": pb.Int(4, endian="big"),
    }
)

data = png.parse("example.png")

which would return a dictionary with the parsed data, like so:

{
     'magic': b'\x89PNG\r\n\x1a\n',
     'ihdr_size': 13,
     'ihdr_id': b'IHDR',
     'width': 602,
     'height': 172,
}

For more real examples, check out the priwo package, which uses pabo to parse pulsar data from binary files (in fact, many of pabo's features are directly motivated by their need in priwo!). Documentation is in development, so stay tuned!

Installation

Installing pabo is as easy as:

pip install pabo

Philosophy

The philosophy behind pabo is: be simple, yet be fast and full of features. This implies that I deliberately avoid coding in features that are too magical or obscure, in contrast to other packages, such as construct. This allows users of pabo to also become contributors, since the internals of pabo are clean and easy-to-understand.