mirror-web-cli
Advanced tools
Professional website mirroring tool with intelligent framework preservation, AI-powered analysis, and comprehensive asset optimization
A powerful, universal website mirroring tool that intelligently detects and preserves framework structures while creating offline-ready websites. Works seamlessly with React, Next.js, Vue, Angular, Svelte, WordPress, and static sites.
🧠 Intelligent Framework Detection
🎨 Beautiful Terminal Experience
⚡ Advanced Asset Processing
🧹 Clean Code Generation
🆕 Auto-Differentiated Output Directories
./domain-standard/ directories./domain-ai-enhanced/ directories✅ Enhanced Environment Variable System
✅ Next.js Image Optimizer Support
/_next/image endpoints with HTTP 402 avoidance✅ Advanced Asset Processing
srcset rewriting✅ Smart Output Organization
# Global installation (recommended)
npm install -g mirror-web-cli
# Or run directly with npx (no installation required)
npx mirror-web-cli https://example.com
For AI-powered website analysis, you'll need an OpenAI API key:
Windows PowerShell:
$env:OPENAI_API_KEY="sk-proj-your-openai-key-here"
Windows Command Prompt:
set OPENAI_API_KEY=sk-proj-your-openai-key-here
macOS/Linux (Bash/Zsh):
export OPENAI_API_KEY="sk-proj-your-openai-key-here"
Permanent Setup (recommended for regular use):
Windows (PowerShell as Administrator):
[System.Environment]::SetEnvironmentVariable('OPENAI_API_KEY', 'sk-proj-your-openai-key-here', 'User')
macOS/Linux (add to ~/.bashrc or ~/.zshrc):
echo 'export OPENAI_API_KEY="sk-proj-your-openai-key-here"' >> ~/.bashrc
source ~/.bashrc
mirror-web-cli https://example.com --ai --openai-key "sk-proj-your-key-here"
Requirements:
sk-)# Standard mirroring (outputs to example.com-standard)
mirror-web-cli https://example.com
# AI-enhanced mirroring (outputs to example.com-ai-enhanced)
mirror-web-cli https://example.com --ai
# Clean mirror without tracking scripts
mirror-web-cli https://react-site.com --clean
# Custom output directory (overrides automatic naming)
mirror-web-cli https://vue-app.com -o ./my-project
# Debug mode with detailed logging
mirror-web-cli https://complex-site.com --debug
Mirror Web CLI automatically creates different output directories based on the analysis method:
./domain-standard (e.g., ./example.com-standard)./domain-ai-enhanced (e.g., ./example.com-ai-enhanced)-o flagThis allows easy comparison between different analysis approaches and organized project management.
# The tool generates a complete project structure
cd ./example.com-standard # or ./example.com-ai-enhanced
# Use any static server to serve the mirrored site
python -m http.server 8000
# Open http://localhost:8000
# Or use Node.js static server
npx serve .
📊 Detection Methods:
├── Script Source Analysis → Framework bundles & runtime files
├── DOM Element Inspection → Framework-specific containers
├── Meta Tag Analysis → Generator tags & signatures
├── Content Pattern Matching → Component structures
├── CSS Class Analysis → Framework styling patterns
├── JSON Data Detection → State management structures
└── Link Href Analysis → Framework asset paths
🎯 Asset Categories:
├── 🖼️ Images → src, srcset, lazy attributes, backgrounds
├── 🎨 Stylesheets → External CSS + inline styles with url() rewriting
├── ⚙️ Scripts → External JS + inline scripts (with optional cleaning)
├── 🔠 Fonts → Web fonts and icon fonts
├── 🎭 Icons → Favicons and app icons
└── 🎥 Media → Videos (.mp4, .webm, .ogg, .avi, .mov, etc.), audio files
📁 Output Structure:
website.com/
├── index.html # Main page with framework intact
├── package.json # Project metadata & serve scripts
├── README.md # Usage instructions
├── server.js # Optional Node.js static server
└── assets/
├── images/ # All images with optimized names
├── css/ # Stylesheets with localized assets
├── js/ # JavaScript files (cleaned if --clean)
├── fonts/ # Web fonts and typography
├── icons/ # Favicons and app icons
└── media/ # Videos (.mp4, .webm, .ogg), audio files, and other media
Modern sites often use:
/_next/image?url=<original>&w=<size>&q=<quality>https://api.microlink.io/?url=... returning either JSON or direct imagesThis tool:
/_next/image directly (avoids 402s)url= param and downloads that/_next/image?... to the same local file as the originalsrc, href, poster, inline style background-imagesrcset and imagesrcset (browsers prefer srcset over src)Verification
Run with --debug and open DevTools Console
Interact with the page (e.g., hover “Preview” links)
Look for lines like:
[MW rewrite] imagesrcset: /_next/image?url=... -> ./assets/images/asset_dc814d3448.png 1x, ...
Open the local asset path (e.g., http://localhost:8000/assets/images/asset_dc814d3448.png)
Blank hover/popover preview
srcset/imagesrcset are being rewritten (use --debug)HTTP 402 from Next.js /_next/image
url=Helpful snippet to locate candidates:
document.querySelectorAll('img, [style]').forEach(n => {
const src = n.currentSrc || n.getAttribute('src') || '';
const styleAttr = n.getAttribute('style') || '';
const bg = getComputedStyle(n).backgroundImage || '';
const hay = [src, styleAttr, bg].join(' ');
if (/(microlink|_next\/image|og|twitter|card)/i.test(hay)) {
console.log('el:', n, { src, styleAttr, bg });
}
});
Usage: mirror-web-cli <url> [options]
Arguments:
url Target website URL to mirror
Options:
-o, --output <dir> Custom output directory (default: domain name)
--clean Remove tracking scripts and analytics
--ai Enable AI-powered analysis (requires OpenAI API key)
--openai-key <key> OpenAI API key for AI features (or set OPENAI_API_KEY env var)
--debug Enable detailed debug logging
--timeout <ms> Page load timeout in milliseconds (default: 120000)
--headless <bool> Run browser in headless mode (default: true)
-h, --help Show help information
-V, --version Show version number
The tool checks for OpenAI API keys in this order:
--openai-key command line parameterOPENAI_API_KEY environment variablesk- (validated automatically)| Framework | Detection | Preservation | Output Quality |
|---|---|---|---|
| React | ✅ High confidence | ✅ Component structure | ⭐⭐⭐⭐⭐ |
| Next.js | ✅ Advanced patterns | ✅ SSR/SSG structure | ⭐⭐⭐⭐⭐ |
| Vue.js | ✅ Reactive patterns | ✅ Template structure | ⭐⭐⭐⭐⭐ |
| Nuxt | ✅ SSR detection | ✅ Module organization | ⭐⭐⭐⭐⭐ |
| Angular | ✅ Component analysis | ✅ Module structure | ⭐⭐⭐⭐⭐ |
| Svelte | ✅ Store patterns | ✅ Component logic | ⭐⭐⭐⭐⭐ |
| Gatsby | ✅ GraphQL detection | ✅ Static generation | ⭐⭐⭐⭐⭐ |
| WordPress | ✅ Theme detection | ✅ Content structure | ⭐⭐⭐⭐ |
| Static Sites | ✅ Always works | ✅ Clean HTML/CSS/JS | ⭐⭐⭐⭐⭐ |
# Simple static site
mirror-web-cli https://example.com
# → Creates: ./example.com-standard/ with complete offline functionality
# React SPA with complex routing
mirror-web-cli https://react-app.com --clean
# → Creates: ./react-app.com-standard/ preserves React structure, removes tracking, offline-ready
# Next.js with image optimization and error handling
mirror-web-cli https://nextjs-site.com --clean
# → Creates: ./nextjs-site.com-standard/ with enhanced Next.js compatibility
# → Handles /_next/image URLs, fixes hydration issues, preserves SSR structure
# Complex site with lots of assets
mirror-web-cli https://shop.example.com --debug --clean
# → Creates: ./shop.example.com-standard/ with detailed logging, removes analytics
Windows PowerShell:
# Set environment variable first
$env:OPENAI_API_KEY="sk-proj-your-openai-key-here"
mirror-web-cli https://complex-app.com --ai --clean
# → Creates: ./complex-app.com-ai-enhanced/ with OpenAI GPT-4o framework analysis
macOS/Linux:
# Set environment variable first
export OPENAI_API_KEY="sk-proj-your-openai-key-here"
mirror-web-cli https://complex-app.com --ai --clean
# → Creates: ./complex-app.com-ai-enhanced/ with OpenAI GPT-4o framework analysis
Cross-platform (using CLI parameter):
# Compare standard vs AI-enhanced outputs
mirror-web-cli https://react-app.com --clean # → ./react-app.com-standard/
mirror-web-cli https://react-app.com --ai --clean # → ./react-app.com-ai-enhanced/
# Mirror for development reference
mirror-web-cli https://design-system.com -o ./reference
cd ./reference
npm start # Built-in development server
# Websites with hero videos (like VS Code, Apple, etc.)
mirror-web-cli https://code.visualstudio.com --clean
# → Downloads all video formats (.mp4, .webm), preserves video posters
# → Handles responsive video sources with media queries
# → Supports autoplay, muted, and poster attributes
# Complex video embedding
mirror-web-cli https://video-heavy-site.com --timeout 180000
# → Extended timeout for large video downloads
# → Maintains video element structure and JavaScript controls
════════════════════════════════════════════════════════════════════════════════
🪞 Mirror Web CLI v1.1.3
Professional Website Mirroring
════════════════════════════════════════════════════════════════════════════════
✨ Features:
• Intelligent framework detection (React, Vue, Angular, Next.js, etc.)
• Framework-preserving output with professional structure
• Comprehensive asset extraction and optimization
• Clean code generation with tracking script removal
🚀 Quick Start:
mirror-web-cli https://example.com
mirror-web-cli https://react-app.com --clean -o ./my-project
╭──────────────────────────────────────────────────────────────────────────────╮
● Step 3/7 • Framework Analysis
Detecting technology stack and framework patterns...
╰──────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────────────────────────────────╮
📦 Framework Analysis
Framework: Next.js
Confidence: 95% ████████████████████░
Complexity: HIGH
Strategy: Preserve DOM; localize assets for exact Next.js look
╰──────────────────────────────────────────────────────────────────────────────╯
src/
├── cli.js # Command-line interface & argument parsing
├── core/ # Core functionality modules
│ ├── mirror-cloner.js # Main orchestrator class
│ ├── browser-engine.js # Puppeteer browser management
│ ├── framework-analyzer.js # Intelligent framework detection
│ ├── asset-manager.js # Comprehensive asset extraction
│ ├── framework-writer.js # Output generation & structure
│ ├── display.js # Beautiful terminal UI system
│ ├── logger.js # Logging & warning management
│ ├── file-writer.js # File system operations
│ ├── filename-utils.js # Smart filename generation
│ └── server.js # Optional static server
└── ai/ # AI-powered analysis (optional)
└── ai-analyzer.js # OpenAI integration for analysis
// In src/core/framework-analyzer.js
this.frameworks.myframework = {
name: 'My Framework',
patterns: [
{ type: 'script', pattern: /myframework\.js/ },
{ type: 'element', selector: '#my-app' },
{ type: 'meta', name: 'generator', pattern: /myframework/i }
]
};
// In src/core/asset-manager.js
async extractCustomAssets() {
// Add your custom asset extraction logic
}
We welcome contributions! Here's how to get started:
# Development setup
git clone https://github.com/SanjeevSaniel/mirror-web-cli.git
cd mirror-web-cli
npm install
# Run tests
npm test
# Development with debugging
npm run dev -- https://example.com --debug
--debug flag for detailed error information--timeout 180000 (3 minutes)--debug to see detection processWindows PowerShell "export command not found":
# ❌ Wrong (Bash syntax)
export OPENAI_API_KEY="sk-..."
# ✅ Correct (PowerShell syntax)
$env:OPENAI_API_KEY="sk-..."
Windows Command Prompt:
# ✅ Correct (CMD syntax)
set OPENAI_API_KEY=sk-your-key-here
Verify environment variable is set:
# PowerShell
echo $env:OPENAI_API_KEY
# Command Prompt
echo %OPENAI_API_KEY%
# Bash/Zsh
echo $OPENAI_API_KEY
sk---debug to see AI analysis processIframe-based sites (like hitesh.ai):
Some sites are just iframe wrappers pointing to external URLs
Example: hitesh.ai loads hiteshchoudhary.com in an iframe
Solution: Mirror the actual content site directly:
# Instead of the wrapper
mirror-web-cli https://hitesh.ai
# Mirror the actual content
mirror-web-cli https://hiteshchoudhary.com --clean
Sites with heavy JavaScript dependencies:
Some React/Next.js sites may need additional processing
Try AI-enhanced mode for better framework handling:
mirror-web-cli https://your-site.com --ai --clean
--debug flag for detailed loggingSpecial thanks to the amazing open-source community:
MIT License - see LICENSE file for details.
Made with ❤️ by Sanjeev Saniel Kujur
Convert any website to universal HTML/CSS/JS with intelligent framework preservation!
FAQs
Professional website mirroring tool with intelligent framework preservation, AI-powered analysis, and comprehensive asset optimization
We found that mirror-web-cli demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Multiple high-impact npm maintainers confirm they have been targeted in the same social engineering campaign that compromised Axios.
Security News
Axios compromise traced to social engineering, showing how attacks on maintainers can bypass controls and expose the broader software supply chain.
Security News
Node.js has paused its bug bounty program after funding ended, removing payouts for vulnerability reports but keeping its security process unchanged.