order-book

Orderbook

A fast L2/L3 orderbook data structure, in C, for Python

Basic Usage

from decimal import Decimal

import requests
from order_book import OrderBook

ob = OrderBook()

# get some orderbook data
data = requests.get("https://api.pro.coinbase.com/products/BTC-USD/book?level=2").json()

ob.bids = {Decimal(price): size for price, size, _ in data['bids']}
ob.asks = {Decimal(price): size for price, size, _ in data['asks']}

# OR

for side in data:
    # there is additional data we need to ignore
    if side in {'bids', 'asks'}:
        ob[side] = {Decimal(price): size for price, size, _ in data[side]}


# Data is accessible by .index(), which returns a tuple of (price, size) at that level in the book
price, size = ob.bids.index(0)
print(f"Best bid price: {price} size: {size}")

price, size = ob.asks.index(0)
print(f"Best ask price: {price} size: {size}")

print(f"The spread is {ob.asks.index(0)[0] - ob.bids.index(0)[0]}\n\n")

# Data is accessible via iteration
# Note: bids/asks are iterators

print("Bids")
for price in ob.bids:
    print(f"Price: {price} Size: {ob.bids[price]}")


print("\n\nAsks")
for price in ob.asks:
    print(f"Price: {price} Size: {ob.asks[price]}")


# Data can be exported to a sorted dictionary
# In Python3.7+ dictionaries remain in insertion ordering. The
# dict returned by .to_dict() has had its keys inserted in sorted order
print("\n\nRaw asks dictionary")
print(ob.asks.to_dict())


# Data can also be exported as an ordered list
# .to_list() returns a list of (price, size) tuples
print("Top 5 Asks")
print(ob.asks.to_list()[:5])
print("\nTop 5 Bids")
print(ob.bids.to_list()[:5])

Main Features

• Sides maintained in correct order
• Can perform orderbook checksums
• Supports max depth and depth truncation

Installation

The preferable way to install is via pip - pip install order-book. Installing from source will require a compiler and can be done with setuptools: python setup.py install.

Running code coverage

The script coverage.sh will compile the source using the -coverage CFLAG, run the unit tests, and build a coverage report in HTML. The script uses tools that may need to be installed (coverage, lcov, genhtml).

Running the performance tests

You can run the performance tests like so: python perf/performance_test.py. The program will profile the time to run for random data samples of various sizes as well as the construction of a sorted orderbook using live L2 orderbook data from Coinbase.

The performance of constructing a sorted orderbook (using live data from Coinbase) using this C library, versus a pure Python sorted dictionary library:

Library	Time, in seconds
C Library	0.00021767616271
Python Library	0.00043988227844

The performance of constructing sorted dictionaries using the same libraries, as well as the cost of building unsorted, python dictionaies for dictionaries of random floating point data:

Library	Number of Keys	Time, in seconds
C Library	100	0.00021600723266
Python Library	100	0.00044703483581
Python Dict	100	0.00022006034851
C Library	500	0.00103306770324
Python Library	500	0.00222206115722
Python Dict	500	0.00097918510437
C Library	1000	0.00202703475952
Python Library	1000	0.00423812866210
Python Dict	1000	0.00176715850830

This represents a roughly 2x speedup compared to a pure python implementation, and in many cases is close to the performance of an unsorted python dictionary.

For other performance metrics, run performance_test.py as well as the other performance tests in perf/

Changelog

0.6.1 (2024-04-22)

• Update: to_list's behavior matches that of to_dict (respects max_depth, if set).
• Update: resolve build warnings on some compilers.

0.6.0 (2022-10-19)

• Update: Drop support for python 3.7
• Feature: to_list method
• Bugfix: Initialize iterator correctly

0.5.0 (2022-08-23)

• Bugfix: fix segmentation fault when calculating checksum on empty orderbook
• Bugfix: fix missing reference decrement
• Performance: Improvement to marking dirty keys

0.4.3 (2022-05-29)

• Bugfix: handle scientific notation of small values in Kraken checksum
• Update: calculate Kraken checksum on order books less than 10 levels deep
• Bugfix: fix occasional incorrect checksums for OKX, FTX and Bitget

0.4.2 (2022-04-17)

• Update: OKEx renamed OKX (for checksum validation)
• Feature: Add support for orderbook checksums with Bitget

0.4.1 (2021-10-12)

• Bugfix: unnecessary reference counting prevented sorted dictionaries from being deallocated
• Bugfix: setting ordering on a sorted dict before checking that it was created successfully

0.4.0 (2021-09-16)

• Feature: changes to code and setup.py to enable compiling on windows
• Feature: add from_type/to_type kwargs to the to_dict methods, allowing for type conversion when creating the dictionary

0.3.2 (2021-09-04)

• Bugfix: depth was incorrectly ignored when converting sorteddict to python dict

0.3.1 (2021-09-01)

• Bugfix: truncate and max_depth not being passed from orderbook to sorteddict object correctly
• Feature: let checksum_format kwarg be set to None

0.3.0 (2021-07-16)

• Update classifiers to indicate this projects only supports MacOS/Linux
• Bugfix: Using less than the minimum number of levels for a checksum with Kraken not raising error correctly
• Update: add del examples to test code

0.2.1 (2021-03-29)

• Bugfix: Invalid deallocation of python object

0.2.0 (2021-03-12)

• Feature: Add branch prediction hints around error handling code
• Bugfix: Fix regression from adding branch predictors
• Bugfix: Fix error corner case when iterating twice on an empty dataset
• Feature: Add contains function for membership test
• Bugfix: Fix issues around storing L3 data
• Feature: Enhance testing, add in L3 book test cases

0.1.1 (2021-02-12)

• Feature: Checksum support for orderbooks
• Feature: FTX checksum support
• Feature: Kraken checksum support
• Feature: OkEX/OKCoin checksum support
• Perf: Use CRC32 table to improve performance of checksum code

0.1.0 (2021-01-18)

• Minor: Use enums to make code more readable
• Bugfix: Add manifest file to ensure headers and changes file are included in sdist builds
• Feature: Add support for max depth and depth truncation

0.0.2 (2020-12-27)

• Bugfix: Fix sorted dictionary arg parsing
• Feature: Coverage report generation for C library
• Bugfix: Fix reference counting in index method in SortedDict
• Feature: New unit tests to improve SortedDict coverage
• Feature: Modularize files
• Feature: Add ability to set bids/asks to dictionaries via attributes or [ ]
• Docs: Update README with simple usage example

0.0.1 (2020-12-26)

• Initial Release