pyfunc-pipeline

PyFunc: Functional Programming Pipeline for Python

PyFunc is a Python library that brings functional programming fluency to Python, enabling chainable, composable, lazy, and debuggable operations on various data structures.

✨ Features

• 🔗 Chainable Operations: Method chaining for readable data transformations
• 🎯 Placeholder Syntax: Use _ to create lambda-free expressions
• ⚡ Lazy Evaluation: Operations computed only when needed
• 🔄 Function Composition: Compose functions with >> and << operators
• 📊 Rich Data Operations: Works with scalars, lists, dicts, generators
• 🐛 Built-in Debugging: Debug and trace pipeline execution
• 🔧 Extensible: Register custom types and extend functionality
• 📝 Type Safe: Full type hints and generic support
• 🚀 Multi-Backend Performance: 5 backends for optimal performance
  • Python: Universal compatibility (always available)
  • C++: High-performance general operations
  • Rust: Memory-safe statistical functions
  • Go: Lightning-fast bitwise operations
  • Zig: Blazing mathematical computations (236x speedup!)

🚀 Quick Start

pip install pyfunc-pipeline

from pyfunc import pipe, _

# Basic pipeline
result = pipe([1, 2, 3, 4]).filter(_ > 2).map(_ * 10).to_list()
# Result: [30, 40]

# String processing  
result = pipe("hello world").explode(" ").map(_.capitalize()).implode(" ").get()
# Result: "Hello World"

# Function composition
double = _ * 2
square = _ ** 2
composed = double >> square  # square(double(x))

result = pipe(5).apply(composed).get()
# Result: 100

⚡ Performance Backends

from pyfunc import pipe, set_zig_threshold, set_go_threshold

# Configure performance backends
set_zig_threshold(1000)  # Use Zig for math operations ≥ 1000 elements
set_go_threshold(500)    # Use Go for bitwise operations ≥ 500 elements

# Automatic backend selection
large_data = list(range(5000))
result = pipe(large_data).sum().get()  # Uses Zig automatically (blazing fast!)

# Explicit backend control
result = pipe([1, 2, 3, 4, 5]).sum_zig().get()           # Force Zig
result = pipe([15, 31, 63]).bitwise_and_go(7).to_list()  # Force Go
result = pipe([1, 2, 3, 4, 5]).median_rust().get()      # Force Rust

# Batch operations for maximum performance
from pyfunc.backends import get_backend
backend = get_backend()
if backend.zig_backend:
    stats = backend.zig_backend.batch_statistics([1, 2, 3, 4, 5])
    # Returns: {'sum': 15.0, 'mean': 3.0, 'min': 1.0, 'max': 5.0, 'stdev': 1.414}

🎯 Core Concepts

Pipeline Chaining

Every value can be lifted into a pipeline for transformation:

from pyfunc import pipe, _

# Numbers
pipe([1, 2, 3, 4]).filter(_ > 2).map(_ ** 2).sum().get()
# Result: 25

# Strings  
pipe("  hello world  ").apply(_.strip().title()).explode(" ").to_list()
# Result: ['Hello', 'World']

# Dictionaries
pipe({"a": 1, "b": 2}).map_values(_ * 10).get()
# Result: {"a": 10, "b": 20}

Placeholder Syntax

The _ placeholder creates reusable, composable expressions:

from pyfunc import _

# Arithmetic operations
double = _ * 2
add_ten = _ + 10

# Method calls
normalize = _.strip().lower()

# Comparisons  
is_positive = _ > 0

# Composition
process = double >> add_ten  # add_ten(double(x))

Lazy Evaluation

Operations are lazy by default - perfect for large datasets:

# Processes only what's needed from 1 million items
result = pipe(range(1_000_000)).filter(_ > 500_000).take(5).to_list()

📚 Rich API

String Operations

pipe("hello,world").explode(",").map(_.capitalize()).implode(" & ").get()
# "Hello & World"

pipe("Hello {name}!").template_fill({"name": "PyFunc"}).get()  
# "Hello PyFunc!"

Dictionary Operations

users = {"alice": 25, "bob": 30}
pipe(users).map_values(_ + 5).map_keys(_.title()).get()
# {"Alice": 30, "Bob": 35}

Advanced Transformations

# Group by
data = [{"name": "Alice", "dept": "Eng"}, {"name": "Bob", "dept": "Sales"}]
pipe(data).group_by(_["dept"]).get()

# Sliding windows
pipe([1, 2, 3, 4, 5]).window(3).to_list()
# [[1, 2, 3], [2, 3, 4], [3, 4, 5]]

# Combinations
pipe([1, 2, 3]).combinations(2).to_list()  
# [(1, 2), (1, 3), (2, 3)]

Side Effects & Debugging

pipe([1, 2, 3, 4])
    .debug("Input")
    .filter(_ > 2) 
    .debug("Filtered")
    .map(_ ** 2)
    .to_list()

🌟 Real-World Example

from pyfunc import pipe, _

# E-commerce order processing with template mapping
orders = [
    {"id": 1, "customer": "Alice", "items": ["laptop", "mouse"], "total": 1200.50},
    {"id": 2, "customer": "Bob", "items": ["keyboard"], "total": 75.00},
    {"id": 3, "customer": "Charlie", "items": ["monitor", "stand"], "total": 450.25}
]

# Process orders with dictionary and string templates
result = (
    pipe(orders)
    .filter(_["total"] > 100)  # Filter orders > $100
    .map({
        "id": _["id"],
        "customer": _["customer"],
        "discounted_total": _["total"] * 0.9  # 10% discount
    })
    .map("Order #{id} for {customer}: ${discounted_total:.2f}")
    .to_list()
)

print(result)
# ['Order #1 for Alice: $1080.45', 'Order #3 for Charlie: $405.23']

⚠️ Important Note

Use regular string templates, not f-strings:

# ❌ Wrong - Don't use f-strings  
.map(f"Order #{_['id']}")  # This will cause an error!

# ✅ Correct - Use regular string templates
.map("Order #{id}")  # PyFunc handles the evaluation

📖 Documentation

• Complete Documentation - Full API reference and examples
• Examples - Real-world usage examples
• Changelog - Version history and updates

🚀 Performance Backends

• C++ Backend - High-performance general operations
• Zig Backend - Blazing-fast mathematical operations (236x speedup!)
• Rust Backend - Memory-safe statistical functions
• Go Backend - Lightning-fast bitwise operations
• Backend Control Guide - Complete user control examples

🔧 Backend Installation

# Install with all backends (recommended)
pip install pyfunc-pipeline[all]

# Or install specific backends
pip install pyfunc-pipeline[cpp]    # C++ backend
pip install pyfunc-pipeline[zig]    # Zig backend  
pip install pyfunc-pipeline[rust]   # Rust backend
pip install pyfunc-pipeline[go]     # Go backend

# Build backends from source
python build_zig.py    # Build Zig backend
python build_go.py     # Build Go backend
python build_cpp.py    # Build C++ backend

🤝 Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

📄 License

MIT License - see LICENSE file for details.