⚡ Blazing fast Rust-powered utilities to eliminate Python's performance bottlenecks.
pip install pyroidpip install pyroid
For development installation:
git clone https://github.com/ao/pyroid.git
cd pyroid
python build_and_install.py
This script will:
Pyroid provides high-performance implementations across multiple domains:
| Module | Description | Key Functions |
|---|---|---|
| Math | Numerical computations | vector_operations, matrix_operations, statistics |
| String | Text processing | reverse, base64_encode, base64_decode |
| Data | Collection and DataFrame operations | filter, map, reduce, dataframe_apply |
| I/O | File and network operations | read_file, write_file, http_get, http_post |
| Image | Basic image manipulation | create_image, to_grayscale, resize, blur |
| ML | Basic machine learning | kmeans, linear_regression, normalize, distance_matrix |
| Core | Core functionality | runtime, buffer, parallel |
Pyroid uses feature flags to allow selective compilation of components:
| Feature Flag | Description | Default |
|---|---|---|
math | Math operations | Enabled |
text | Text processing | Enabled |
data | Collection and DataFrame operations | Enabled |
io | File and network operations | Enabled |
image | Basic image processing | Enabled |
ml | Basic machine learning | Enabled |
To compile with only specific features, modify your Cargo.toml:
[dependencies]
pyroid = { version = "0.1.0", default-features = false, features = ["math", "data"] }
import pyroid
# Vector operations
v1 = pyroid.math.Vector([1, 2, 3])
v2 = pyroid.math.Vector([4, 5, 6])
v3 = v1 + v2
print(f"Vector sum: {v3}")
print(f"Dot product: {v1.dot(v2)}")
# Matrix operations
m1 = pyroid.math.Matrix([[1, 2], [3, 4]])
m2 = pyroid.math.Matrix([[5, 6], [7, 8]])
m3 = m1 * m2
print(f"Matrix product: {m3}")
# Statistical functions
numbers = [1, 2, 3, 4, 5]
mean = pyroid.math.stats.mean(numbers)
median = pyroid.math.stats.median(numbers)
std_dev = pyroid.math.stats.calc_std(numbers)
print(f"Mean: {mean}, Median: {median}, StdDev: {std_dev}")
import pyroid
# Basic string operations
text = "Hello, world!"
reversed_text = pyroid.text.reverse(text)
uppercase = pyroid.text.to_uppercase(text)
lowercase = pyroid.text.to_lowercase(text)
# Base64 encoding/decoding
encoded = pyroid.text.base64_encode(text)
decoded = pyroid.text.base64_decode(encoded)
print(f"Original: {text}")
print(f"Encoded: {encoded}")
print(f"Decoded: {decoded}")
import pyroid
# Create a DataFrame
df = pyroid.data.DataFrame({
'id': [1, 2, 3, 4, 5],
'name': ['Alice', 'Bob', 'Charlie', 'David', 'Eve'],
'age': [25, 30, 35, 40, 45]
})
# Apply a function to each column
result = pyroid.data.apply(df, lambda x: x * 2, axis=0)
print(f"DataFrame: {df}")
print(f"Applied function: {result}")
# Group by and aggregate
grouped = pyroid.data.groupby_aggregate(df, "age", {"name": "count"})
print(f"Grouped by age: {grouped}")
import pyroid
# Filter a list
numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
even_numbers = pyroid.data.filter(numbers, lambda x: x % 2 == 0)
print(f"Even numbers: {even_numbers}")
# Map a function over a list
squared = pyroid.data.map(numbers, lambda x: x * x)
print(f"Squared numbers: {squared}")
# Reduce a list
sum_result = pyroid.data.reduce(numbers, lambda x, y: x + y)
print(f"Sum: {sum_result}")
# Sort a list
unsorted = [5, 2, 8, 1, 9, 3]
sorted_list = pyroid.data.sort(unsorted, reverse=True) # Note: supports reverse parameter
print(f"Sorted (descending): {sorted_list}")
import pyroid
# Read a file
content = pyroid.io.read_file("example.txt")
print(f"File content length: {len(content)}")
# Write a file
pyroid.io.write_file("output.txt", "Hello, world!") # Note: string, not bytes
# Read multiple files
files = ["file1.txt", "file2.txt", "file3.txt"]
contents = pyroid.io.read_files(files)
print(f"Read multiple files: {contents}")
import pyroid
# Make a GET request
response = pyroid.io.get("https://example.com")
print(f"HTTP GET response length: {len(response)}")
# Note: POST requests might not be implemented in the current version
# If you need to make POST requests, check the documentation for the latest API
import asyncio
import pyroid
from pyroid.core import runtime, buffer, parallel
async def main():
# Initialize the runtime
runtime.init()
print(f"Runtime initialized with {runtime.get_worker_threads()} worker threads")
# Create an AsyncClient for HTTP operations
client = pyroid.AsyncClient()
# Fetch a URL asynchronously
response = await client.fetch("https://example.com")
print(f"Status code: {response['status']}")
# Fetch multiple URLs concurrently
urls = [f"https://example.com/{i}" for i in range(10)]
responses = await client.fetch_many(urls, concurrency=5)
print(f"Fetched {len(responses)} URLs")
# Async file operations
file_reader = pyroid.AsyncFileReader("example.txt")
content = await file_reader.read_all()
print(f"File content length: {len(content)}")
# Zero-copy buffer operations
zero_copy_buffer = buffer.ZeroCopyBuffer(1024) # 1KB buffer
data = zero_copy_buffer.get_data()
# Modify data...
zero_copy_buffer.set_data(data)
# Parallel processing
processor = parallel.BatchProcessor(batch_size=1000, adaptive=True)
items = list(range(1000000))
def process_item(x):
return x * x
results = processor.map(items, process_item)
print(f"Processed {len(results)} items")
# Run the async main function
asyncio.run(main())
For a more comprehensive example, see examples/optimized_async_example.py.
import pyroid
# Create a new image (width, height, channels)
img = pyroid.image.basic.create_image(100, 100, 3)
# Set some pixels
for x in range(50):
for y in range(50):
img.set_pixel(x, y, [255, 0, 0]) # Red square
for x in range(50, 100):
for y in range(50, 100):
img.set_pixel(x, y, [0, 0, 255]) # Blue square
# Apply operations
grayscale_img = img.to_grayscale()
resized_img = img.resize(200, 200)
blurred_img = img.blur(2)
brightened_img = img.adjust_brightness(1.5)
# Get image data
width = img.width
height = img.height
channels = img.channels
data = img.data
import pyroid
# K-means clustering
data = [
[1.0, 2.0], [1.5, 1.8], [5.0, 8.0],
[8.0, 8.0], [1.0, 0.6], [9.0, 11.0]
]
kmeans_result = pyroid.ml.basic.kmeans(data, k=2)
print(f"K-means centroids: {kmeans_result['centroids']}")
print(f"K-means clusters: {kmeans_result['clusters']}")
# Linear regression
# Note: Linear regression expects X as 2D array and y as 1D array
X = [[1, 1], [1, 2], [2, 2], [2, 3]]
y = [6, 8, 9, 11]
regression_result = pyroid.ml.basic.linear_regression(X, y)
print(f"Linear regression coefficients: {regression_result['coefficients']}")
print(f"Linear regression intercept: {regression_result['intercept']}")
print(f"Linear regression R-squared: {regression_result['r_squared']}")
# Data normalization
normalized_data = pyroid.ml.basic.normalize(data, method="min-max")
print(f"Normalized data (min-max): {normalized_data}")
# Distance matrix
distance_matrix = pyroid.ml.basic.distance_matrix(data, metric="euclidean")
print(f"Distance matrix shape: {len(distance_matrix)}x{len(distance_matrix[0])}")
Pyroid offers significant performance improvements over pure Python:
For detailed benchmarks and performance comparisons between Pyroid's Rust implementation and Python alternatives, see our Performance Comparison document. Our benchmarks show that Pyroid's Rust implementation can be up to 15,000x faster than equivalent Python code for certain operations, with performance advantages that scale dramatically with data size.
Note: The Python fallback implementations are provided only for development and testing when the Rust components cannot be built. For production use, the Rust implementation is essential to achieve the performance benefits.
To build pyroid from source, you need:
The easiest way to build and install pyroid is to use the provided script:
python build_and_install.py
Alternatively, you can build manually:
# Build the Rust code
cargo build --release
# Install the Python package in development mode
pip install -e .
For performance-critical applications, consider using the following optimizations:
Verify Rust Implementation: Ensure you're using the high-performance Rust implementation
python check_implementation.py
Run Python Tests: Ensure the Python fallback implementations work correctly
python run_tests.py
Run Rust Tests: Ensure the Rust implementations work correctly
# Run all Rust tests
cargo test --test test_rust_*
# Or run specific module tests
cargo test --test test_rust_core # Core functionality tests
cargo test --test test_rust_math # Math operations tests
cargo test --test test_rust_data # Data operations tests
cargo test --test test_rust_text # Text operations tests
cargo test --test test_rust_io # I/O operations tests
cargo test --test test_rust_image # Image operations tests
cargo test --test test_rust_impl # ML operations tests
For detailed information about the Rust tests, see Rust Tests Documentation.
Unified Runtime: Initialize the runtime once at the start of your application
from pyroid.core import runtime
runtime.init()
Zero-Copy Buffers: Use zero-copy buffers for large data transfers
from pyroid.core import buffer
zero_copy_buffer = buffer.ZeroCopyBuffer(size)
Parallel Processing: Use batch processing for CPU-intensive operations
from pyroid.core import parallel
processor = parallel.BatchProcessor(adaptive=True)
results = processor.map(items, process_function)
Concurrency Control: Adjust concurrency levels based on your workload
client = pyroid.AsyncClient()
responses = await client.fetch_many(urls, concurrency=optimal_value)
To run the benchmarks and see the performance improvements:
# Build and install pyroid first
python build_and_install.py
# Run the async benchmarks
python -m benchmarks.run_benchmarks --size small --suite async --no-dashboard
# Run the high-throughput benchmark
python -m benchmarks.run_benchmarks --size small --suite high-throughput --no-dashboard
To run the example demonstrating all the optimized features:
# Build and install pyroid first
python build_and_install.py
# Run the example
python examples/optimized_async_example.py
MIT
Contributions are welcome! Please feel free to submit a Pull Request.
git checkout -b feature/amazing-feature)git commit -m 'Add some amazing feature')git push origin feature/amazing-feature)FAQs
High-performance Rust functions for Python with a simplified architecture
We found that pyroid demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
More than half of CISOs now manage 10+ security areas, often with few legal safeguards and short tenures, yet continue to secure budgets and higher pay.
Security News
Libxml2’s solo maintainer drops embargoed security fixes, highlighting the burden on unpaid volunteers who keep critical open source software secure.
Research
Security News
Socket investigates hidden protestware in npm packages that blocks user interaction and plays the Ukrainian anthem for Russian-language visitors.