sideloader

🚀 Sideload

Download large files via PyPI packages! Sideload automatically splits large files into PyPI-compliant chunks and allows you to download them through a beautiful CLI interface.

Features

• ✨ Beautiful CLI powered by Rich with progress bars and status updates
• 📦 Automatic file splitting into 99MB PyPI-compliant chunks
• 🔄 Automatic reassembly of downloaded parts
• 🌐 JSONBin integration for request tracking
• 🛡️ Error handling with detailed progress monitoring

Installation

git clone <repository-url>
cd Sideload
uv install

Usage

Environment Variables

Set up your credentials:

export JSONBIN_TOKEN="your_jsonbin_token"
export SIDELOAD_COLLECTION_ID="your_collection_id"

Download a File

# Basic usage
uv run sideload download https://example.com/largefile.zip

# Specify output directory
uv run sideload download https://example.com/largefile.zip --output ./downloads/

# Override credentials
uv run sideload download https://example.com/largefile.zip --token YOUR_TOKEN --collection YOUR_COLLECTION

How it Works

• Submit Request: The CLI creates a new request in your JSONBin collection
• Monitor Progress: Real-time progress monitoring with beautiful progress bars
• Download Packages: Automatically downloads all PyPI packages containing file parts
• Reassemble: Extracts and concatenates parts to rebuild the original file

CLI Interface

The CLI provides:

• 🌈 Colorful output with status indicators
• 📊 Progress bars for downloads and processing
• 📈 Real-time monitoring of server-side processing
• ✅ Success/error reporting with detailed information
• 📋 Summary tables showing download statistics

Example Output

🚀 SIDELOAD
Download large files via PyPI packages

🌐 Requesting download for: https://example.com/largefile.zip
✅ Created sideload request: abc123def456

📡 Monitoring Progress
📥 Downloading... (45%) ████████████████▌                 
🔨 Building packages...
📤 Uploading part 1/3...

📊 Download Summary
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property          ┃ Value                           ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Original Filename │ largefile.zip                   │
│ File Size         │ 250,123,456 bytes               │
│ Total Packages    │ 3                               │
│ Status            │ ✅ UPLOADED                     │
└───────────────────┴─────────────────────────────────┘

📦 Downloading Packages
📦 Downloading package 1/3... ████████████████████████████

🔧 Reassembling File
🔗 Assembling part 1/3... ████████████████████████████

✨ Complete
🎉 File successfully downloaded to: largefile.zip
📊 File size: 250,123,456 bytes

Development

Server Setup

The server component handles file processing and PyPI uploads:

# Set environment variables
export JSONBIN_TOKEN="your_token"
export PYPI_TOKEN="your_pypi_token"

# Run the server
uv run python src/sideloader/main.py

Kubernetes Deployment

Before installing the Helm chart, create the required secret in your namespace:

kubectl create secret generic sideload-secrets \
  --namespace <namespace> \
  --from-literal=jsonbin-token="YOUR_JSONBIN_TOKEN" \
  --from-literal=pypi-token="YOUR_PYPI_TOKEN" \
  --from-literal=pypi-user="YOUR_PYPI_USERNAME" \
  --from-literal=pypi-password="YOUR_PYPI_PASSWORD" \
  --from-literal=pypi-totp="YOUR_PYPI_TOTP_SECRET"  # optional, omit if not using TOTP

Then install the chart:

helm install sideload ./helm --namespace <namespace>

Project Structure

src/sideloader/
├── __init__.py           # Package initialization
├── cli.py                # CLI client
├── server.py             # Server component
├── jsonbin_connector.py  # JSONBin API wrapper
├── pypi_cleanup.py       # PyPI package deletion via Playwright
└── scripts/
    └── cleanup_pypi.py   # Admin script to delete all sideload packages

License

[Your License Here]