Computing dot-products, similarity measures, and distances between low- and high-dimensional vectors is ubiquitous in Machine Learning, Scientific Computing, Geo-Spatial Analysis, and Information Retrieval.
These algorithms generally have linear complexity in time, constant or linear complexity in space, and are data-parallel.
In other words, it is easily parallelizable and vectorizable and often available in packages like BLAS (level 1) and LAPACK, as well as higher-level numpy and scipy Python libraries.
Ironically, even with decades of evolution in compilers and numerical computing, most libraries can be 3-200x slower than hardware potential even on the most popular hardware, like 64-bit x86 and Arm CPUs.
Moreover, most lack mixed-precision support, which is crucial for modern AI!
The rare few that support minimal mixed precision, run only on one platform, and are vendor-locked, by companies like Intel and Nvidia.
SimSIMD provides an alternative.
1️⃣ SimSIMD functions are practically as fast as memcpy.
2️⃣ Unlike BLAS, most kernels are designed for mixed-precision and bit-level operations.
3️⃣ SimSIMD often ships more binaries than NumPy and has more backends than most BLAS implementations, and more high-level interfaces than most libraries.
SimSIMD (Arabic: "سيمسيم دي") is a mixed-precision math library of over 350 SIMD-optimized kernels extensively used in AI, Search, and DBMS workloads. Named after the iconic "Open Sesame" command that opened doors to treasure in Ali Baba and the Forty Thieves, SimSimd can help you 10x the cost-efficiency of your computational pipelines. Implemented distance functions include:
Moreover, SimSIMD...
float64, float32, float16, and bfloat16 real & complex vectors.int8 integral, int4 sub-byte, and b8 binary vectors.uint32 and uint16 sets, and weighted sparse vectors.Due to the high-level of fragmentation of SIMD support in different x86 CPUs, SimSIMD generally uses the names of select Intel CPU generations for its backends. They, however, also work on AMD CPUs. Intel Haswell is compatible with AMD Zen 1/2/3, while AMD Genoa Zen 4 covers AVX-512 instructions added to Intel Skylake and Ice Lake. You can learn more about the technical implementation details in the following blog-posts:
for-loops.sqrt with Newton Raphson iterations.PyArg_ParseTuple using faster calling convention.| NumPy | C 99 | SimSIMD |
|---|---|---|
cosine distances between 1536d vectors in int8 | ||
|
🚧 overflows |
x86: 10,548,600 ops/s arm: 11,379,300 ops/s |
x86: 16,151,800 ops/s arm: 13,524,000 ops/s |
cosine distances between 1536d vectors in bfloat16 | ||
|
🚧 not supported |
x86: 119,835 ops/s arm: 403,909 ops/s |
x86: 9,738,540 ops/s arm: 4,881,900 ops/s |
cosine distances between 1536d vectors in float16 | ||
|
x86: 40,481 ops/s arm: 21,451 ops/s |
x86: 501,310 ops/s arm: 871,963 ops/s |
x86: 7,627,600 ops/s arm: 3,316,810 ops/s |
cosine distances between 1536d vectors in float32 | ||
|
x86: 253,902 ops/s arm: 46,394 ops/s |
x86: 882,484 ops/s arm: 399,661 ops/s |
x86: 8,202,910 ops/s arm: 3,400,620 ops/s |
cosine distances between 1536d vectors in float64 | ||
|
x86: 212,421 ops/s arm: 52,904 ops/s |
x86: 839,301 ops/s arm: 837,126 ops/s |
x86: 1,538,530 ops/s arm: 1,678,920 ops/s |
eculidean distance between 1536d vectors in int8 | ||
|
x86: 252,113 ops/s arm: 177,443 ops/s |
x86: 6,690,110 ops/s arm: 4,114,160 ops/s |
x86: 18,989,000 ops/s arm: 18,878,200 ops/s |
eculidean distance between 1536d vectors in bfloat16 | ||
|
🚧 not supported |
x86: 119,842 ops/s arm: 1,049,230 ops/s |
x86: 9,727,210 ops/s arm: 4,233,420 ops/s |
eculidean distance between 1536d vectors in float16 | ||
|
x86: 54,621 ops/s arm: 71,793 ops/s |
x86: 196,413 ops/s arm: 911,370 ops/s |
x86: 19,466,800 ops/s arm: 3,522,760 ops/s |
eculidean distance between 1536d vectors in float32 | ||
|
x86: 424,944 ops/s arm: 292,629 ops/s |
x86: 1,295,210 ops/s arm: 1,055,940 ops/s |
x86: 8,924,100 ops/s arm: 3,602,650 ops/s |
eculidean distance between 1536d vectors in float64 | ||
|
x86: 334,929 ops/s arm: 237,505 ops/s |
x86: 1,215,190 ops/s arm: 905,782 ops/s |
x86: 1,701,740 ops/s arm: 1,735,840 ops/s |
For benchmarks we mostly use 1536-dimensional vectors, like the embeddings produced by the OpenAI Ada API. The code was compiled with GCC 12, using glibc v2.35. The benchmarks performed on Arm-based Graviton3 AWS
c7ginstances andr7izIntel Sapphire Rapids. Most modern Arm-based 64-bit CPUs will have similar relative speedups. Variance withing x86 CPUs will be larger.
Similar speedups are often observed even when compared to BLAS and LAPACK libraries underlying most numerical computing libraries, including NumPy and SciPy in Python. Broader benchmarking results:
The package is intended to replace the usage of numpy.inner, numpy.dot, and scipy.spatial.distance.
Aside from drastic performance improvements, SimSIMD significantly improves accuracy in mixed precision setups.
NumPy and SciPy, processing int8, uint8 or float16 vectors, will use the same types for accumulators, while SimSIMD can combine int8 enumeration, int16 multiplication, and int32 accumulation to avoid overflows entirely.
The same applies to processing float16 and bfloat16 values with float32 precision.
Use the following snippet to install SimSIMD and list available hardware acceleration options available on your machine:
pip install simsimd
python -c "import simsimd; print(simsimd.get_capabilities())" # for hardware introspection
python -c "import simsimd; help(simsimd)" # for documentation
With precompiled binaries, SimSIMD ships .pyi interface files for type hinting and static analysis.
You can check all the available functions in python/annotations/__init__.pyi.
import simsimd
import numpy as np
vec1 = np.random.randn(1536).astype(np.float32)
vec2 = np.random.randn(1536).astype(np.float32)
dist = simsimd.cosine(vec1, vec2)
Supported functions include cosine, inner, sqeuclidean, hamming, jaccard, kulbackleibler, jensenshannon, and intersect.
Dot products are supported for both real and complex numbers:
vec1 = np.random.randn(768).astype(np.float64) + 1j * np.random.randn(768).astype(np.float64)
vec2 = np.random.randn(768).astype(np.float64) + 1j * np.random.randn(768).astype(np.float64)
dist = simsimd.dot(vec1.astype(np.complex128), vec2.astype(np.complex128))
dist = simsimd.dot(vec1.astype(np.complex64), vec2.astype(np.complex64))
dist = simsimd.vdot(vec1.astype(np.complex64), vec2.astype(np.complex64)) # conjugate, same as `np.vdot`
Unlike SciPy, SimSIMD allows explicitly stating the precision of the input vectors, which is especially useful for mixed-precision setups.
The dtype argument can be passed both by name and as a positional argument:
dist = simsimd.cosine(vec1, vec2, "int8")
dist = simsimd.cosine(vec1, vec2, "float16")
dist = simsimd.cosine(vec1, vec2, "float32")
dist = simsimd.cosine(vec1, vec2, "float64")
dist = simsimd.hamming(vec1, vec2, "bin8")
Binary distance functions are computed at a bit-level.
Meaning a vector of 10x 8-bit integers will be treated as a sequence of 80 individual bits or dimensions.
This differs from NumPy, that can't handle smaller-than-byte types, but you can still avoid the bin8 argument by reinterpreting the vector as booleans:
vec1 = np.random.randint(2, size=80).astype(np.uint8).packbits().view(np.bool_)
vec2 = np.random.randint(2, size=80).astype(np.uint8).packbits().view(np.bool_)
hamming_distance = simsimd.hamming(vec1, vec2)
jaccard_distance = simsimd.jaccard(vec1, vec2)
With other frameworks, like PyTorch, one can get a richer type-system than NumPy, but the lack of good CPython interoperability makes it hard to pass data without copies.
Here is an example of using SimSIMD with PyTorch to compute the cosine similarity between two bfloat16 vectors:
import numpy as np
buf1 = np.empty(8, dtype=np.uint16)
buf2 = np.empty(8, dtype=np.uint16)
# View the same memory region with PyTorch and randomize it
import torch
vec1 = torch.asarray(memoryview(buf1), copy=False).view(torch.bfloat16)
vec2 = torch.asarray(memoryview(buf2), copy=False).view(torch.bfloat16)
torch.randn(8, out=vec1)
torch.randn(8, out=vec2)
# Both libs will look into the same memory buffers and report the same results
dist_slow = 1 - torch.nn.functional.cosine_similarity(vec1, vec2, dim=0)
dist_fast = simsimd.cosine(buf1, buf2, "bfloat16")
It also allows using SimSIMD for half-precision complex numbers, which NumPy does not support.
For that, view data as continuous even-length np.float16 vectors and override type-resolution with complex32 string.
vec1 = np.random.randn(1536).astype(np.float16)
vec2 = np.random.randn(1536).astype(np.float16)
simd.dot(vec1, vec2, "complex32")
simd.vdot(vec1, vec2, "complex32")
When dealing with sparse representations and integer sets, you can apply the intersect function to two 1-dimensional arrays of uint16 or uint32 integers:
from random import randint
import numpy as np
import simsimd as simd
length1, length2 = randint(1, 100), randint(1, 100)
vec1 = np.sort(np.random.randint(0, 1000, length1).astype(np.uint16))
vec2 = np.sort(np.random.randint(0, 1000, length2).astype(np.uint16))
slow_result = len(np.intersect1d(vec1, vec2))
fast_result = simd.intersect(vec1, vec2)
assert slow_result == fast_result
Every distance function can be used not only for one-to-one but also one-to-many and many-to-many distance calculations. For one-to-many:
vec1 = np.random.randn(1536).astype(np.float32) # rank 1 tensor
batch1 = np.random.randn(1, 1536).astype(np.float32) # rank 2 tensor
batch2 = np.random.randn(100, 1536).astype(np.float32)
dist_rank1 = simsimd.cosine(vec1, batch2)
dist_rank2 = simsimd.cosine(batch1, batch2)
All distance functions in SimSIMD can be used to compute many-to-many distances. For two batches of 100 vectors to compute 100 distances, one would call it like this:
batch1 = np.random.randn(100, 1536).astype(np.float32)
batch2 = np.random.randn(100, 1536).astype(np.float32)
dist = simsimd.cosine(batch1, batch2)
Input matrices must have identical shapes. This functionality isn't natively present in NumPy or SciPy, and generally requires creating intermediate arrays, which is inefficient and memory-consuming.
One can use SimSIMD to compute distances between all possible pairs of rows across two matrices (akin to scipy.spatial.distance.cdist).
The resulting object will have a type DistancesTensor, zero-copy compatible with NumPy and other libraries.
For two arrays of 10 and 1,000 entries, the resulting tensor will have 10,000 cells:
import numpy as np
from simsimd import cdist, DistancesTensor
matrix1 = np.random.randn(1000, 1536).astype(np.float32)
matrix2 = np.random.randn(10, 1536).astype(np.float32)
distances: DistancesTensor = simsimd.cdist(matrix1, matrix2, metric="cosine") # zero-copy, managed by SimSIMD
distances_array: np.ndarray = np.array(distances, copy=True) # now managed by NumPy
SimSIMD also provides mixed-precision element-wise kernels, where the input vectors and the output have the same numeric type, but the intermediate accumulators are of a higher precision.
import numpy as np
from simsimd import fma, wsum
# Let's take two FullHD video frames
first_frame = np.random.randn(1920 * 1024).astype(np.uint8)
second_frame = np.random.randn(1920 * 1024).astype(np.uint8)
average_frame = np.empty_like(first_frame)
wsum(first_frame, second_frame, alpha=0.5, beta=0.5, out=average_frame)
# Slow analog with NumPy:
slow_average_frame = (0.5 * first_frame + 0.5 * second_frame).astype(np.uint8)
Similarly, the fma takes three arguments and computes the fused multiply-add operation.
In applications like Machine Learning you may also benefit from using the "brain-float" format not natively supported by NumPy.
In 3D Graphics, for example, we can use FMA to compute the Phong shading model:
# Assume a FullHD frame with random values for simplicity
light_intensity = np.random.rand(1920 * 1080).astype(np.float16) # Intensity of light on each pixel
diffuse_component = np.random.rand(1920 * 1080).astype(np.float16) # Diffuse reflectance on the surface
specular_component = np.random.rand(1920 * 1080).astype(np.float16) # Specular reflectance for highlights
output_color = np.empty_like(light_intensity) # Array to store the resulting color intensity
# Define the scaling factors for diffuse and specular contributions
alpha = 0.7 # Weight for the diffuse component
beta = 0.3 # Weight for the specular component
# Formula: color = alpha * light_intensity * diffuse_component + beta * specular_component
fma(light_intensity, diffuse_component, specular_component,
dtype="float16", # Optional, unless it can't be inferred from the input
alpha=alpha, beta=beta, out=output_color)
# Slow analog with NumPy for comparison
slow_output_color = (alpha * light_intensity * diffuse_component + beta * specular_component).astype(np.float16)
By default, computations use a single CPU core.
To override this behavior, use the threads argument.
Set it to 0 to use all available CPU cores.
Here is an example of dealing with large sets of binary vectors:
ndim = 1536 # OpenAI Ada embeddings
matrix1 = np.packbits(np.random.randint(2, size=(10_000, ndim)).astype(np.uint8))
matrix2 = np.packbits(np.random.randint(2, size=(1_000, ndim)).astype(np.uint8))
distances = simsimd.cdist(matrix1, matrix2,
metric="hamming", # Unlike SciPy, SimSIMD doesn't divide by the number of dimensions
out_dtype="uint8", # so we can use `uint8` instead of `float64` to save memory.
threads=0, # Use all CPU cores with OpenMP.
dtype="bin8", # Override input argument type to `bin8` eight-bit words.
)
By default, the output distances will be stored in double-precision float64 floating-point numbers.
That behavior may not be space-efficient, especially if you are computing the hamming distance between short binary vectors, that will generally fit into 8x smaller uint8 or uint16 types.
To override this behavior, use the dtype argument.
You can turn specific backends on or off depending on the exact environment. A common case may be avoiding AVX-512 on older AMD CPUs and Intel Ice Lake CPUs to ensure the CPU doesn't change the frequency license and throttle performance.
$ simsimd.get_capabilities()
> {'serial': True, 'neon': False, 'sve': False, 'neon_f16': False, 'sve_f16': False, 'neon_bf16': False, 'sve_bf16': False, 'neon_i8': False, 'sve_i8': False, 'haswell': True, 'skylake': True, 'ice': True, 'genoa': True, 'sapphire': True, 'turin': True}
$ simsimd.disable_capability("sapphire")
$ simsimd.enable_capability("sapphire")
Want to use it in Python with USearch?
You can wrap the raw C function pointers SimSIMD backends into a CompiledMetric and pass it to USearch, similar to how it handles Numba's JIT-compiled code.
from usearch.index import Index, CompiledMetric, MetricKind, MetricSignature
from simsimd import pointer_to_sqeuclidean, pointer_to_cosine, pointer_to_inner
metric = CompiledMetric(
pointer=pointer_to_cosine("f16"),
kind=MetricKind.Cos,
signature=MetricSignature.ArrayArraySize,
)
index = Index(256, metric=metric)
To install, add the following to your Cargo.toml:
[dependencies]
simsimd = "..."
Before using the SimSIMD library, ensure you have imported the necessary traits and types into your Rust source file.
The library provides several traits for different distance/similarity kinds - SpatialSimilarity, BinarySimilarity, and ProbabilitySimilarity.
use simsimd::SpatialSimilarity;
fn main() {
let vector_a: Vec<f32> = vec![1.0, 2.0, 3.0];
let vector_b: Vec<f32> = vec![4.0, 5.0, 6.0];
// Compute the cosine similarity between vector_a and vector_b
let cosine_similarity = f32::cosine(&vector_a, &vector_b)
.expect("Vectors must be of the same length");
println!("Cosine Similarity: {}", cosine_similarity);
// Compute the squared Euclidean distance between vector_a and vector_b
let sq_euclidean_distance = f32::sqeuclidean(&vector_a, &vector_b)
.expect("Vectors must be of the same length");
println!("Squared Euclidean Distance: {}", sq_euclidean_distance);
}
Spatial similarity functions are available for f64, f32, f16, and i8 types.
use simsimd::SpatialSimilarity;
use simsimd::ComplexProducts;
fn main() {
let vector_a: Vec<f32> = vec![1.0, 2.0, 3.0, 4.0];
let vector_b: Vec<f32> = vec![5.0, 6.0, 7.0, 8.0];
// Compute the inner product between vector_a and vector_b
let inner_product = SpatialSimilarity::dot(&vector_a, &vector_b)
.expect("Vectors must be of the same length");
println!("Inner Product: {}", inner_product);
// Compute the complex inner product between complex_vector_a and complex_vector_b
let complex_inner_product = ComplexProducts::dot(&vector_a, &vector_b)
.expect("Vectors must be of the same length");
let complex_conjugate_inner_product = ComplexProducts::vdot(&vector_a, &vector_b)
.expect("Vectors must be of the same length");
println!("Complex Inner Product: {:?}", complex_inner_product); // -18, 69
println!("Complex C. Inner Product: {:?}", complex_conjugate_inner_product); // 70, -8
}
Complex inner products are available for f64, f32, and f16 types.
use simsimd::SpatialSimilarity;
fn main() {
let vector_a: Vec<f32> = vec![1.0, 2.0, 3.0];
let vector_b: Vec<f32> = vec![4.0, 5.0, 6.0];
let cosine_similarity = f32::jensenshannon(&vector_a, &vector_b)
.expect("Vectors must be of the same length");
println!("Cosine Similarity: {}", cosine_similarity);
let sq_euclidean_distance = f32::kullbackleibler(&vector_a, &vector_b)
.expect("Vectors must be of the same length");
println!("Squared Euclidean Distance: {}", sq_euclidean_distance);
}
Probability similarity functions are available for f64, f32, and f16 types.
Similar to spatial distances, one can compute bit-level distance functions between slices of unsigned integers:
use simsimd::BinarySimilarity;
fn main() {
let vector_a = &[0b11110000, 0b00001111, 0b10101010];
let vector_b = &[0b11110000, 0b00001111, 0b01010101];
// Compute the Hamming distance between vector_a and vector_b
let hamming_distance = u8::hamming(&vector_a, &vector_b)
.expect("Vectors must be of the same length");
println!("Hamming Distance: {}", hamming_distance);
// Compute the Jaccard distance between vector_a and vector_b
let jaccard_distance = u8::jaccard(&vector_a, &vector_b)
.expect("Vectors must be of the same length");
println!("Jaccard Distance: {}", jaccard_distance);
}
Binary similarity functions are available only for u8 types.
Rust has no native support for half-precision floating-point numbers, but SimSIMD provides a f16 type.
It has no functionality - it is a transparent wrapper around u16 and can be used with half or any other half-precision library.
use simsimd::SpatialSimilarity;
use simsimd::f16 as SimF16;
use half::f16 as HalfF16;
fn main() {
let vector_a: Vec<HalfF16> = ...
let vector_b: Vec<HalfF16> = ...
let buffer_a: &[SimF16] = unsafe { std::slice::from_raw_parts(a_half.as_ptr() as *const SimF16, a_half.len()) };
let buffer_b: &[SimF16] = unsafe { std::slice::from_raw_parts(b_half.as_ptr() as *const SimF16, b_half.len()) };
// Compute the cosine similarity between vector_a and vector_b
let cosine_similarity = SimF16::cosine(&vector_a, &vector_b)
.expect("Vectors must be of the same length");
println!("Cosine Similarity: {}", cosine_similarity);
}
The "brain-float-16" is a popular machine learning format.
It's broadly supported in hardware and is very machine-friendly, but software support is still lagging behind.
Unlike NumPy, you can already use bf16 datatype in SimSIMD.
Luckily, to downcast f32 to bf16 you only have to drop the last 16 bits:
import numpy as np
import simsimd as simd
a = np.random.randn(ndim).astype(np.float32)
b = np.random.randn(ndim).astype(np.float32)
# NumPy doesn't natively support brain-float, so we need a trick!
# Luckily, it's very easy to reduce the representation accuracy
# by simply masking the low 16-bits of our 32-bit single-precision
# numbers. We can also add `0x8000` to round the numbers.
a_f32rounded = ((a.view(np.uint32) + 0x8000) & 0xFFFF0000).view(np.float32)
b_f32rounded = ((b.view(np.uint32) + 0x8000) & 0xFFFF0000).view(np.float32)
# To represent them as brain-floats, we need to drop the second half
a_bf16 = np.right_shift(a_f32rounded.view(np.uint32), 16).astype(np.uint16)
b_bf16 = np.right_shift(b_f32rounded.view(np.uint32), 16).astype(np.uint16)
# Now we can compare the results
expected = np.inner(a_f32rounded, b_f32rounded)
result = simd.inner(a_bf16, b_bf16, "bf16")
SimSIMD provides a dynamic dispatch mechanism to select the most advanced micro-kernel for the current CPU.
You can query supported backends and use the SimSIMD::capabilities function to select the best one.
println!("uses neon: {}", capabilities::uses_neon());
println!("uses sve: {}", capabilities::uses_sve());
println!("uses haswell: {}", capabilities::uses_haswell());
println!("uses skylake: {}", capabilities::uses_skylake());
println!("uses ice: {}", capabilities::uses_ice());
println!("uses genoa: {}", capabilities::uses_genoa());
println!("uses sapphire: {}", capabilities::uses_sapphire());
println!("uses turin: {}", capabilities::uses_turin());
To install, choose one of the following options depending on your environment:
npm install --save simsimdyarn add simsimdpnpm add simsimdbun install simsimdThe package is distributed with prebuilt binaries, but if your platform is not supported, you can build the package from the source via npm run build.
This will automatically happen unless you install the package with the --ignore-scripts flag or use Bun.
After you install it, you will be able to call the SimSIMD functions on various TypedArray variants:
const { sqeuclidean, cosine, inner, hamming, jaccard } = require('simsimd');
const vectorA = new Float32Array([1.0, 2.0, 3.0]);
const vectorB = new Float32Array([4.0, 5.0, 6.0]);
const distance = sqeuclidean(vectorA, vectorB);
console.log('Squared Euclidean Distance:', distance);
Other numeric types and precision levels are supported as well.
For double-precision floating-point numbers, use Float64Array:
const vectorA = new Float64Array([1.0, 2.0, 3.0]);
const vectorB = new Float64Array([4.0, 5.0, 6.0]);
const distance = cosine(vectorA, vectorB);
When doing machine learning and vector search with high-dimensional vectors you may want to quantize them to 8-bit integers.
You may want to project values from the $[-1, 1]$ range to the $[-127, 127]$ range and then cast them to Int8Array:
const quantizedVectorA = new Int8Array(vectorA.map(v => (v * 127)));
const quantizedVectorB = new Int8Array(vectorB.map(v => (v * 127)));
const distance = cosine(quantizedVectorA, quantizedVectorB);
A more extreme quantization case would be to use binary vectors.
You can map all positive values to 1 and all negative values and zero to 0, packing eight values into a single byte.
After that, Hamming and Jaccard distances can be computed.
const { toBinary, hamming } = require('simsimd');
const binaryVectorA = toBinary(vectorA);
const binaryVectorB = toBinary(vectorB);
const distance = hamming(binaryVectorA, binaryVectorB);
To install, simply add the following dependency to you Package.swift:
dependencies: [
.package(url: "https://github.com/ashvardanian/simsimd")
]
The package provides the most common spatial metrics for Int8, Float16, Float32, and Float64 vectors.
import SimSIMD
let vectorA: [Int8] = [1, 2, 3]
let vectorB: [Int8] = [4, 5, 6]
let cosineSimilarity = vectorA.cosine(vectorB) // Computes the cosine similarity
let dotProduct = vectorA.dot(vectorB) // Computes the dot product
let sqEuclidean = vectorA.sqeuclidean(vectorB) // Computes the squared Euclidean distance
For integration within a CMake-based project, add the following segment to your CMakeLists.txt:
FetchContent_Declare(
simsimd
GIT_REPOSITORY https://github.com/ashvardanian/simsimd.git
GIT_SHALLOW TRUE
)
FetchContent_MakeAvailable(simsimd)
After that, you can use the SimSIMD library in your C code in several ways. Simplest of all, you can include the headers, and the compiler will automatically select the most recent CPU extensions that SimSIMD will use.
#include <simsimd/simsimd.h>
int main() {
simsimd_f32_t vector_a[1536];
simsimd_f32_t vector_b[1536];
simsimd_kernel_punned_t distance_function = simsimd_metric_punned(
simsimd_metric_cos_k, // Metric kind, like the angular cosine distance
simsimd_datatype_f32_k, // Data type, like: f16, f32, f64, i8, b8, and complex variants
simsimd_cap_any_k); // Which CPU capabilities are we allowed to use
simsimd_distance_t distance;
distance_function(vector_a, vector_b, 1536, &distance);
return 0;
}
To avoid hard-coding the backend, you can rely on c/lib.c to prepackage all possible backends in one binary, and select the most recent CPU features at runtime.
That feature of the C library is called dynamic dispatch and is extensively used in the Python, JavaScript, and Rust bindings.
To test which CPU features are available on the machine at runtime, use the following APIs:
int uses_dynamic_dispatch = simsimd_uses_dynamic_dispatch(); // Check if dynamic dispatch was enabled
simsimd_capability_t capabilities = simsimd_capabilities(); // Returns a bitmask
int uses_neon = simsimd_uses_neon();
int uses_sve = simsimd_uses_sve();
int uses_haswell = simsimd_uses_haswell();
int uses_skylake = simsimd_uses_skylake();
int uses_ice = simsimd_uses_ice();
int uses_genoa = simsimd_uses_genoa();
int uses_sapphire = simsimd_uses_sapphire();
To override compilation settings and switch between runtime and compile-time dispatch, define the following macro:
#define SIMSIMD_DYNAMIC_DISPATCH 1 // or 0
#include <simsimd/simsimd.h>
int main() {
simsimd_i8_t i8[1536];
simsimd_i8_t u8[1536];
simsimd_f64_t f64s[1536];
simsimd_f32_t f32s[1536];
simsimd_f16_t f16s[1536];
simsimd_bf16_t bf16s[1536];
simsimd_distance_t distance;
// Cosine distance between two vectors
simsimd_cos_i8(i8s, i8s, 1536, &distance);
simsimd_cos_u8(u8s, u8s, 1536, &distance);
simsimd_cos_f16(f16s, f16s, 1536, &distance);
simsimd_cos_f32(f32s, f32s, 1536, &distance);
simsimd_cos_f64(f64s, f64s, 1536, &distance);
simsimd_cos_bf16(bf16s, bf16s, 1536, &distance);
// Euclidean distance between two vectors
simsimd_l2sq_i8(i8s, i8s, 1536, &distance);
simsimd_l2sq_u8(u8s, u8s, 1536, &distance);
simsimd_l2sq_f16(f16s, f16s, 1536, &distance);
simsimd_l2sq_f32(f32s, f32s, 1536, &distance);
simsimd_l2sq_f64(f64s, f64s, 1536, &distance);
simsimd_l2sq_bf16(bf16s, bf16s, 1536, &distance);
return 0;
}
#include <simsimd/simsimd.h>
int main() {
// SimSIMD provides "sized" type-aliases without relying on `stdint.h`
simsimd_i8_t i8[1536];
simsimd_i8_t u8[1536];
simsimd_f16_t f16s[1536];
simsimd_f32_t f32s[1536];
simsimd_f64_t f64s[1536];
simsimd_bf16_t bf16s[1536];
simsimd_distance_t product;
// Inner product between two real vectors
simsimd_dot_i8(i8s, i8s, 1536, &product);
simsimd_dot_u8(u8s, u8s, 1536, &product);
simsimd_dot_f16(f16s, f16s, 1536, &product);
simsimd_dot_f32(f32s, f32s, 1536, &product);
simsimd_dot_f64(f64s, f64s, 1536, &product);
simsimd_dot_bf16(bf16s, bf16s, 1536, &product);
// SimSIMD provides complex types with `real` and `imag` fields
simsimd_f64c_t f64s[768];
simsimd_f32c_t f32s[768];
simsimd_f16c_t f16s[768];
simsimd_bf16c_t bf16s[768];
simsimd_distance_t products[2]; // real and imaginary parts
// Complex inner product between two vectors
simsimd_dot_f16c(f16cs, f16cs, 768, &products[0]);
simsimd_dot_f32c(f32cs, f32cs, 768, &products[0]);
simsimd_dot_f64c(f64cs, f64cs, 768, &products[0]);
simsimd_dot_bf16c(bf16cs, bf16cs, 768, &products[0]);
// Complex conjugate inner product between two vectors
simsimd_vdot_f16c(f16cs, f16cs, 768, &products[0]);
simsimd_vdot_f32c(f32cs, f32cs, 768, &products[0]);
simsimd_vdot_f64c(f64cs, f64cs, 768, &products[0]);
simsimd_vdot_bf16c(bf16cs, bf16cs, 768, &products[0]);
return 0;
}
#include <simsimd/simsimd.h>
int main() {
simsimd_b8_t b8s[1536 / 8]; // 8 bits per word
simsimd_distance_t distance;
simsimd_hamming_b8(b8s, b8s, 1536 / 8, &distance);
simsimd_jaccard_b8(b8s, b8s, 1536 / 8, &distance);
return 0;
}
#include <simsimd/simsimd.h>
int main() {
simsimd_f64_t f64s[1536];
simsimd_f32_t f32s[1536];
simsimd_f16_t f16s[1536];
simsimd_distance_t divergence;
// Jensen-Shannon divergence between two vectors
simsimd_js_f16(f16s, f16s, 1536, &divergence);
simsimd_js_f32(f32s, f32s, 1536, &divergence);
simsimd_js_f64(f64s, f64s, 1536, &divergence);
// Kullback-Leibler divergence between two vectors
simsimd_kl_f16(f16s, f16s, 1536, &divergence);
simsimd_kl_f32(f32s, f32s, 1536, &divergence);
simsimd_kl_f64(f64s, f64s, 1536, &divergence);
return 0;
}
If you aim to utilize the _Float16 functionality with SimSIMD, ensure your development environment is compatible with C 11.
For other SimSIMD functionalities, C 99 compatibility will suffice.
To explicitly disable half-precision support, define the following macro before imports:
#define SIMSIMD_NATIVE_F16 0 // or 1
#define SIMSIMD_NATIVE_BF16 0 // or 1
#include <simsimd/simsimd.h>
SIMSIMD_DYNAMIC_DISPATCH:
By default, SimSIMD is a header-only library. But if you are running on different generations of devices, it makes sense to pre-compile the library for all supported generations at once, and dispatch at runtime. This flag does just that and is used to produce the
simsimd.soshared library, as well as the Python and other bindings.
For Arm: SIMSIMD_TARGET_NEON, SIMSIMD_TARGET_SVE, SIMSIMD_TARGET_SVE2, SIMSIMD_TARGET_NEON_F16, SIMSIMD_TARGET_SVE_F16, SIMSIMD_TARGET_NEON_BF16, SIMSIMD_TARGET_SVE_BF16.
For x86: (SIMSIMD_TARGET_HASWELL, SIMSIMD_TARGET_SKYLAKE, SIMSIMD_TARGET_ICE, SIMSIMD_TARGET_GENOA, SIMSIMD_TARGET_SAPPHIRE, SIMSIMD_TARGET_TURIN, SIMSIMD_TARGET_SIERRA.
By default, SimSIMD automatically infers the target architecture and pre-compiles as many kernels as possible. In some cases, you may want to explicitly disable some of the kernels. Most often it's due to compiler support issues, like the lack of some recent intrinsics or low-precision numeric types. In other cases, you may want to disable some kernels to speed up the compilation process and trim the binary size.
SIMSIMD_SQRT, SIMSIMD_RSQRT, SIMSIMD_LOG:
By default, for non-SIMD backends, SimSIMD may use
libcfunctions likesqrtandlog. Those are generally very accurate, but slow, and introduce a dependency on the C standard library. To avoid that you can override those definitions with your custom implementations, like:#define SIMSIMD_RSQRT(x) (1 / sqrt(x)).
In general there are a few principles that SimSIMD follows:
errno.Possibly, in the future:
NaN components in low-precision inputs.NaN.Last, but not the least - don't build unless there is a demand for it. So if you have a specific use-case, please open an issue or a pull request, and ideally, bring in more users with similar needs.
The cosine similarity is the most common and straightforward metric used in machine learning and information retrieval. Interestingly, there are multiple ways to shoot yourself in the foot when computing it. The cosine similarity is the inverse of the cosine distance, which is the cosine of the angle between two vectors.
\text{CosineSimilarity}(a, b) = \frac{a \cdot b}{\|a\| \cdot \|b\|}
\text{CosineDistance}(a, b) = 1 - \frac{a \cdot b}{\|a\| \cdot \|b\|}
In NumPy terms, SimSIMD implementation is similar to:
import numpy as np
def cos_numpy(a: np.ndarray, b: np.ndarray) -> float:
ab, a2, b2 = np.dot(a, b), np.dot(a, a), np.dot(b, b) # Fused in SimSIMD
if a2 == 0 and b2 == 0: result = 0 # Same in SciPy
elif ab == 0: result = 1 # Division by zero error in SciPy
else: result = 1 - ab / (sqrt(a2) * sqrt(b2)) # Bigger rounding error in SciPy
return result
In SciPy, however, the cosine distance is computed as 1 - ab / np.sqrt(a2 * b2).
It handles the edge case of a zero and non-zero argument pair differently, resulting in a division by zero error.
It's not only less efficient, but also less accurate, given how the reciprocal square roots are computed.
The C standard library provides the sqrt function, which is generally very accurate, but slow.
The rsqrt in-hardware implementations are faster, but have different accuracy characteristics.
rsqrtps and AVX vrsqrtps: $1.5 \times 2^{-12}$ maximal relative error.vrsqrt14pd instruction: $2^{-14}$ maximal relative error.frsqrte instruction has no documented error bounds, but can be $2^{-3}$.To overcome the limitations of the rsqrt instruction, SimSIMD uses the Newton-Raphson iteration to refine the initial estimate for high-precision floating-point numbers.
It can be defined as:
x_{n+1} = x_n \cdot (3 - x_n \cdot x_n) / 2
On 1536-dimensional inputs on Intel Sapphire Rapids CPU a single such iteration can result in a 2-3 orders of magnitude relative error reduction:
| Datatype | NumPy Error | SimSIMD w/out Iteration | SimSIMD |
|---|---|---|---|
bfloat16 | 1.89e-08 ± 1.59e-08 | 3.07e-07 ± 3.09e-07 | 3.53e-09 ± 2.70e-09 |
float16 | 1.67e-02 ± 1.44e-02 | 2.68e-05 ± 1.95e-05 | 2.02e-05 ± 1.39e-05 |
float32 | 2.21e-08 ± 1.65e-08 | 3.47e-07 ± 3.49e-07 | 3.77e-09 ± 2.84e-09 |
float64 | 0.00e+00 ± 0.00e+00 | 3.80e-07 ± 4.50e-07 | 1.35e-11 ± 1.85e-11 |
The Mahalanobis distance is a generalization of the Euclidean distance, which takes into account the covariance of the data. It's very similar in its form to the bilinear form, which is a generalization of the dot product.
\text{BilinearForm}(a, b, M) = a^T M b
\text{Mahalanobis}(a, b, M) = \sqrt{(a - b)^T M^{-1} (a - b)}
Bilinear Forms can be seen as one of the most important linear algebraic operations, surprisingly missing in BLAS and LAPACK. They are versatile and appear in various domains:
Broad applications aside, the lack of a specialized primitive for bilinear forms in BLAS and LAPACK means significant performance overhead. A $vector * matrix * vector$ product is a scalar, whereas its constituent parts ($vector * matrix$ and $matrix * vector$) are vectors:
SimSIMD doesn't produce intermediate vector results, like a @ M @ b, but computes the bilinear form directly.
The set intersection operation is generally defined as the number of elements that are common between two sets, represented as sorted arrays of integers. The most common way to compute it is a linear scan:
size_t intersection_size(int *a, int *b, size_t n, size_t m) {
size_t i = 0, j = 0, count = 0;
while (i < n && j < m) {
if (a[i] < b[j]) i++;
else if (a[i] > b[j]) j++;
else i++, j++, count++;
}
return count;
}
Alternatively, one can use the binary search to find the elements in the second array that are present in the first one. On every step the checked region of the second array is halved, which is called the galloping search. It's faster, but only when large arrays of very different sizes are intersected. Third approach is to use the SIMD instructions to compare multiple elements at once:
pcmpestrm.vp2intersectd.After benchmarking, the last approach was chosen, as it's the most flexible and often the fastest.
Complex dot products are a generalization of the dot product to complex numbers.
They are supported by most BLAS packages, but almost never in mixed precision.
SimSIMD defines dot and vdot kernels as:
\text{dot}(a, b) = \sum_{i=0}^{n-1} a_i \cdot b_i
\text{vdot}(a, b) = \sum_{i=0}^{n-1} a_i \cdot \bar{b_i}
Where $\bar{b_i}$ is the complex conjugate of $b_i$. Putting that into Python code for scalar arrays:
def dot(a: List[number], b: List[number]) -> number:
a_real, a_imaginary = a[0::2], a[1::2]
b_real, b_imaginary = b[0::2], b[1::2]
ab_real, ab_imaginary = 0, 0
for ar, ai, br, bi in zip(a_real, a_imaginary, b_real, b_imaginary):
ab_real += ar * br - ai * bi
ab_imaginary += ar * bi + ai * br
return ab_real, ab_imaginary
def vdot(a: List[number], b: List[number]) -> number:
a_real, a_imaginary = a[0::2], a[1::2]
b_real, b_imaginary = b[0::2], b[1::2]
ab_real, ab_imaginary = 0, 0
for ar, ai, br, bi in zip(a_real, a_imaginary, b_real, b_imaginary):
ab_real += ar * br + ai * bi
ab_imaginary += ar * bi - ai * br
return ab_real, ab_imaginary
The Kullback-Leibler divergence is a measure of how one probability distribution diverges from a second, expected probability distribution. Jensen-Shannon divergence is a symmetrized and smoothed version of the Kullback-Leibler divergence, which can be used as a distance metric between probability distributions.
\text{KL}(P || Q) = \sum_{i} P(i) \log \frac{P(i)}{Q(i)}
\text{JS}(P, Q) = \frac{1}{2} \text{KL}(P || M) + \frac{1}{2} \text{KL}(Q || M), M = \frac{P + Q}{2}
Both functions are defined for non-negative numbers, and the logarithm is a key part of their computation.
The Fused-Multiply-Add (FMA) operation is a single operation that combines element-wise multiplication and addition with different scaling factors. The Weighted Sum is it's simplified variant without element-wise multiplication.
\text{FMA}_i(A, B, C, \alpha, \beta) = \alpha \cdot A_i \cdot B_i + \beta \cdot C_i
\text{WSum}_i(A, B, \alpha, \beta) = \alpha \cdot A_i + \beta \cdot B_i
In NumPy terms, the implementation may look like:
import numpy as np
def wsum(A: np.ndarray, B: np.ndarray, /, Alpha: float, Beta: float) -> np.ndarray:
assert A.dtype == B.dtype, "Input types must match and affect the output style"
return (Alpha * A + Beta * B).astype(A.dtype)
def fma(A: np.ndarray, B: np.ndarray, C: np.ndarray, /, Alpha: float, Beta: float) -> np.ndarray:
assert A.dtype == B.dtype and A.dtype == C.dtype, "Input types must match and affect the output style"
return (Alpha * A * B + Beta * C).astype(A.dtype)
The tricky part is implementing those operations in mixed precision, where the scaling factors are of different precision than the input and output vectors.
SimSIMD uses double-precision floating-point scaling factors for any input and output precision, including i8 and u8 integers and f16 and bf16 floats.
Depending on the generation of the CPU, given native support for f16 addition and multiplication, the f16 temporaries are used for i8 and u8 multiplication, scaling, and addition.
For bf16, native support is generally limited to dot-products with subsequent partial accumulation, which is not enough for the FMA and WSum operations, so f32 is used as a temporary.
On the Intel Sapphire Rapids platform, SimSIMD was benchmarked against auto-vectorized code using GCC 12.
GCC handles single-precision float but might not be the best choice for int8 and _Float16 arrays, which have been part of the C language since 2011.
| Kind | GCC 12 f32 | GCC 12 f16 | SimSIMD f16 | f16 improvement |
|---|---|---|---|---|
| Inner Product | 3,810 K/s | 192 K/s | 5,990 K/s | 31 x |
| Cosine Distance | 3,280 K/s | 336 K/s | 6,880 K/s | 20 x |
| Euclidean Distance ² | 4,620 K/s | 147 K/s | 5,320 K/s | 36 x |
| Jensen-Shannon Divergence | 1,180 K/s | 18 K/s | 2,140 K/s | 118 x |
Most popular software is precompiled and distributed with fairly conservative CPU optimizations, to ensure compatibility with older hardware. Database Management platforms, like ClickHouse, and Web Browsers, like Google Chrome,need to run on billions of devices, and they can't afford to be picky about the CPU features. For such users SimSIMD provides a dynamic dispatch mechanism, which selects the most advanced micro-kernel for the current CPU at runtime.
| Subset | F | CD | ER | PF | 4FMAPS | 4VNNIW | VPOPCNTDQ | VL | DQ | BW | IFMA | VBMI | VNNI | BF16 | VBMI2 | BITALG | VPCLMULQDQ | GFNI | VAES | VP2INTERSECT | FP16 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Knights Landing (Xeon Phi x200, 2016) | Yes | Yes | No | ||||||||||||||||||
| Knights Mill (Xeon Phi x205, 2017) | Yes | No | |||||||||||||||||||
| Skylake-SP, Skylake-X (2017) | No | No | Yes | No | |||||||||||||||||
| Cannon Lake (2018) | Yes | No | |||||||||||||||||||
| Cascade Lake (2019) | No | Yes | No | ||||||||||||||||||
| Cooper Lake (2020) | Yes | No | |||||||||||||||||||
| Ice Lake (2019) | Yes | No | Yes | No | |||||||||||||||||
| Tiger Lake (2020) | Yes | No | |||||||||||||||||||
| Rocket Lake (2021) | No | ||||||||||||||||||||
| Alder Lake (2021) | Partial | Partial | |||||||||||||||||||
| Zen 4 (2022) | Yes | Yes | No | ||||||||||||||||||
| Sapphire Rapids (2023) | No | Yes | |||||||||||||||||||
| Zen 5 (2024) | Yes | No |
You can compile SimSIMD on an old CPU, like Intel Haswell, and run it on a new one, like AMD Genoa, and it will automatically use the most advanced instructions available.
Reverse is also true, you can compile on a new CPU and run on an old one, and it will automatically fall back to the most basic instructions.
Moreover, the very first time you prove for CPU capabilities with simsimd_capabilities(), it initializes the dynamic dispatch mechanism, and all subsequent calls will be faster and won't face race conditions in multi-threaded environments.
SimSIMD exposes all kernels for all backends, and you can select the most advanced one for the current CPU without relying on built-in dispatch mechanisms.
That's handy for testing and benchmarking, but also in case you want to dispatch a very specific kernel for a very specific CPU, bypassing SimSIMD assignment logic.
All of the function names follow the same pattern: simsimd_{function}_{type}_{backend}.
serial, haswell, skylake, ice, genoa, sapphire, turin, neon, or sve.f64, f32, f16, bf16, f64c, f32c, f16c, bf16c, i8, or b8.dot, vdot, cos, l2sq, hamming, jaccard, kl, js, or intersect.To avoid hard-coding the backend, you can use the simsimd_kernel_punned_t to pun the function pointer and the simsimd_capabilities function to get the available backends at runtime.
To match all the function names, consider a RegEx:
SIMSIMD_PUBLIC void simsimd_\w+_\w+_\w+\(
On Linux, you can use the following command to list all unique functions:
$ grep -oP 'SIMSIMD_PUBLIC void simsimd_\w+_\w+_\w+\(' include/simsimd/*.h | sort | uniq
> include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_hamming_b8_haswell(
> include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_hamming_b8_ice(
> include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_hamming_b8_neon(
> include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_hamming_b8_serial(
> include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_hamming_b8_sve(
> include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_jaccard_b8_haswell(
> include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_jaccard_b8_ice(
> include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_jaccard_b8_neon(
> include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_jaccard_b8_serial(
> include/simsimd/binary.h:SIMSIMD_PUBLIC void simsimd_jaccard_b8_sve(
FAQs
Portable mixed-precision BLAS-like vector math library for x86 and ARM
We found that simsimd demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
PyPI confirms no security flaws were exploited in the Ultralytics supply chain attack and highlights improvements for safer package publishing.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.