Modular Python tool for profiling files, analyzing directory structures, and inspecting image data
filoma is a modular Python tool for profiling files, analyzing directory structures, and inspecting image data (e.g., .tif, .png, .npy, .zarr). It provides detailed reports on filename patterns, inconsistencies, file counts, empty folders, file system metadata, and image data statistics. The project is designed for easy expansion, testing, CI/CD, Dockerization, and database integration.
# 🚀 RECOMMENDED: Using uv (modern, fast Python package manager)
# Install uv first if you don't have it: curl -LsSf https://astral.sh/uv/install.sh | sh
# For uv projects (recommended - manages dependencies in pyproject.toml):
uv add filoma
# For scripts or non-project environments:
uv pip install filoma
# Traditional method:
pip install filoma
# For maximum performance, also install Rust toolchain:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env
# Then reinstall to build Rust extension:
uv add filoma --force # or: uv pip install --force-reinstall filoma
Note: Rust installation is optional. filoma works perfectly with pure Python, but gets 5-20x faster with Rust acceleration.
uv add filoma → Use this if you have a pyproject.toml file (most Python projects)uv pip install filoma → Use for standalone scripts or when you don't want project dependency managementpip install filoma → Traditional method for older Python environmentsrich).filoma provides a real-time progress bar and timing details for directory analysis, making it easy to track progress on large scans. The progress bar is enabled by default and uses the rich library for beautiful terminal output.
Example:
from filoma.directories import DirectoryProfiler
profiler = DirectoryProfiler(show_progress=True)
result = profiler.analyze("/path/to/large/directory")
profiler.print_summary(result)
# Output includes a progress bar and timing details:
#
# Directory Analysis: /path/to/large/directory (🦀 Rust) - 0.12s
# ┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
# ┃ Metric ┃ Value ┃
# ...
# ┃ Analysis Time ┃ 0.12s ┃
# ┃ Processing Speed ┃ 8,000 items/s ┃
# ┗━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━┛
Performance Note:
The progress bar introduces minimal overhead (especially when updated every 100 items, as in the default implementation). For benchmarking or maximum speed, you can disable it with
show_progress=False.
filoma includes automatic Rust acceleration for directory analysis:
# Install Rust (one-time setup)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env
# Install filoma with Rust acceleration
uv add filoma # For uv projects (recommended)
# or: uv pip install filoma # For scripts/non-project environments
# or: pip install filoma # Traditional method
# The Rust extension builds automatically during installation!
from filoma.directories import DirectoryProfiler
profiler = DirectoryProfiler()
# The output shows which backend is used:
# "Directory Analysis: /path (🦀 Rust)" or "Directory Analysis: /path (🐍 Python)"
result = profiler.analyze("/large/directory")
# Typical speedups:
# - Small dirs (<1K files): 2-5x faster
# - Medium dirs (1K-10K files): 5-10x faster
# - Large dirs (>10K files): 10-20x faster
No code changes needed - your existing code automatically gets faster! 🎉
from filoma.directories import DirectoryProfiler
profiler = DirectoryProfiler()
result = profiler.analyze(".")
# Look for the 🦀 Rust emoji in the report title:
profiler.print_summary(result)
# Output shows: "Directory Analysis: . (🦀 Rust)" or "Directory Analysis: . (🐍 Python)"
# Or check programmatically:
print(f"Rust acceleration: {'✅ Active' if profiler.use_rust else '❌ Not available'}")
import filoma
from filoma.directories import DirectoryProfiler
# Check version and basic functionality
print(f"filoma version: {filoma.__version__}")
profiler = DirectoryProfiler()
print(f"Rust acceleration: {'✅ Active' if profiler.use_rust else '❌ Not available'}")
Pro tip:
- Working on a project? → Use
uv add filoma(manages yourpyproject.tomlautomatically)- Running standalone scripts? → Use
uv pip install filoma- Need compatibility? → Use
pip install filoma- Want the fastest experience? → Install
uvfirst!
from filoma.directories import DirectoryProfiler
# Automatically uses Rust acceleration when available (🦀 Rust)
# Falls back to Python implementation when needed (🐍 Python)
profiler = DirectoryProfiler()
result = profiler.analyze("/path/to/directory", max_depth=3)
# Print comprehensive report with rich formatting
# The report title shows which backend was used!
profiler.print_report(result)
# Or access specific data
print(f"Total files: {result['summary']['total_files']}")
print(f"Total folders: {result['summary']['total_folders']}")
print(f"Empty folders: {result['summary']['empty_folder_count']}")
print(f"File extensions: {result['file_extensions']}")
print(f"Common folder names: {result['common_folder_names']}")
from filoma.directories import DirectoryProfiler
from filoma import DataFrame
# Enable DataFrame building for advanced analysis
profiler = DirectoryProfiler(build_dataframe=True)
result = profiler.analyze("/path/to/directory")
# Get the DataFrame with all file paths
df = profiler.get_dataframe(result)
print(f"Found {len(df)} paths")
# Add path components (parent, name, stem, suffix)
df_enhanced = df.add_path_components()
print(df_enhanced.head())
# Filter by file type
python_files = df.filter_by_extension('.py')
image_files = df.filter_by_extension(['.jpg', '.png', '.tif'])
# Group and analyze
extension_counts = df.group_by_extension()
directory_counts = df.group_by_directory()
# Add file statistics
df_with_stats = df.add_file_stats() # size, timestamps, etc.
# Add depth information
df_with_depth = df.add_depth_column()
# Export for further analysis
df.save_csv("file_analysis.csv")
df.save_parquet("file_analysis.parquet")
from filoma.files import FileProfiler
profiler = FileProfiler()
report = profiler.profile("/path/to/file.txt")
profiler.print_report(report) # Rich table output in your terminal
# Output: (Rich table with file metadata and access rights)
from filoma.images import PngProfiler
profiler = PngProfiler()
report = profiler.analyze("/path/to/image.png")
print(report)
# Output: {'shape': ..., 'dtype': ..., 'min': ..., 'max': ..., 'nans': ..., ...}
The DirectoryProfiler provides comprehensive analysis of directory structures:
max_depth parameterWhen enabled with build_dataframe=True, you get access to powerful data analysis capabilities:
{
"root_path": "/analyzed/path",
"summary": {
"total_files": 150,
"total_folders": 25,
"total_size_bytes": 1048576,
"total_size_mb": 1.0,
"avg_files_per_folder": 6.0,
"max_depth": 3,
"empty_folder_count": 2
},
"file_extensions": {".py": 45, ".txt": 30, ".md": 10},
"common_folder_names": {"src": 3, "tests": 2, "docs": 1},
"empty_folders": ["/path/to/empty1", "/path/to/empty2"],
"top_folders_by_file_count": [("/path/with/most/files", 25)],
"depth_distribution": {0: 1, 1: 5, 2: 12, 3: 7},
"dataframe": filoma.DataFrame # When build_dataframe=True
}
The filoma.DataFrame class provides:
# Path manipulation
df.add_path_components() # Add parent, name, stem, suffix columns
df.add_depth_column() # Add directory depth column
df.add_file_stats() # Add size, timestamps, file type info
# Filtering
df.filter_by_extension('.py') # Filter by single extension
df.filter_by_extension(['.jpg', '.png']) # Filter by multiple extensions
df.filter_by_pattern('test') # Filter by path pattern
# Analysis
df.group_by_extension() # Group and count by file extension
df.group_by_directory() # Group and count by parent directory
# Export
df.save_csv("analysis.csv") # Export to CSV
df.save_parquet("analysis.parquet") # Export to Parquet
df.to_polars() # Get underlying Polars DataFrame
src/filoma/directories/ — Directory analysis and structure profilingsrc/filoma/images/ — Image profilers and analysissrc/filoma/files/ — File profiling (system metadata)tests/ — All tests (unit, integration, and scripts) are in this folderFor users who want to understand or customize the Rust acceleration:
walkdir crate# Force Python implementation (useful for debugging)
profiler = DirectoryProfiler(use_rust=False)
# Check which backend is being used
print(f"Using Rust: {profiler.use_rust}")
# Compare performance
import time
start = time.time()
result = profiler.analyze("/path/to/directory")
print(f"Analysis took {time.time() - start:.3f}s")
filoma is under active development. Contributions and suggestions are welcome!
FAQs
Modular Python tool for profiling files, analyzing directory structures, and inspecting image data
We found that filoma demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
/Security News
A flawed sandbox in @nestjs/devtools-integration lets attackers run code on your machine via CSRF, leading to full Remote Code Execution (RCE).
Product
Customize license detection with Socket’s new license overlays: gain control, reduce noise, and handle edge cases with precision.
Product
Socket now supports Rust and Cargo, offering package search for all users and experimental SBOM generation for enterprise projects.