git-ingest

Git-Ingest 🚀

A powerful CLI tool for analyzing and ingesting project codebases into structured text files with advanced gitignore support, binary file detection, and cross-platform clipboard integration.

🚀 Installation

# Global installation
npm install -g git-ingest

# Or use with npx
npx git-ingest

📖 Usage

Basic Usage

# Analyze current directory
git-ingest

# Analyze specific directory
git-ingest /path/to/project

# Copy output to clipboard
git-ingest --copy

# Verbose output with progress
git-ingest --verbose

# Quiet mode
git-ingest --quiet

Advanced Options

# Custom output filename (Markdown)
git-ingest --output my-project-analysis.md

# Generate classic text format output
git-ingest --format text --output analysis.txt

# Include only specific file patterns
git-ingest --include "*.js" "*.ts" "*.json"

# Exclude specific patterns
git-ingest --exclude "*.test.js" "*.spec.js"

# Set maximum file size (in MB)
git-ingest --max-size 5

# Generate text format with custom output
git-ingest --format text --output analysis.txt

All Options

git-ingest [options] [directory]

Arguments:
  directory                    Target directory to analyze (default: "./")

Options:
  -V, --version                output the version number
  -o, --output <filename>      Specify output filename
  -f, --format <type>          Output format: markdown or text (default: "markdown")
  -c, --copy                   Copy output to clipboard
  -i, --include <patterns...>  Include files matching patterns
  -e, --exclude <patterns...>  Exclude files matching patterns
  --max-size <size>            Maximum file size to include (in MB) (default: "10")
  -v, --verbose                Verbose output
  -q, --quiet                  Quiet mode
  -h, --help                   display help for command

📊 Output Format

Git-Ingest supports two output formats optimized for different use cases. Markdown is now the default output format.

📝 Markdown Format (Default, LLM-Optimized)

The default output is a structured, semantic Markdown format designed for optimal AI/LLM processing:

git-ingest

Features:

• 🎯 LLM-Friendly Structure: Hierarchical organization with proper semantic markup
• 🔍 Language Detection: Automatic syntax highlighting for 60+ programming languages
• 📊 Rich Metadata: File statistics, language distribution, and categorization
• 📑 Table of Contents: Navigation-friendly structure with links
• 🏷️ File Categorization: Organized by Web Frontend, Backend, DevOps, etc.
• 💻 Code Blocks: Proper syntax highlighting for better readability
• 📈 Analytics: Comprehensive project statistics and insights

Example markdown output structure:

# 🚀 Project Analysis Report

## 📊 Project Overview

**Language:** javascript | **Files:** 25 | **Size:** 2.3 MB

## 📈 Statistics

### 📊 File Type Distribution

| Category       | Files | Percentage |
| -------------- | ----- | ---------- |
| Web Frontend   | 15    | 60.0%      |
| Backend/Server | 8     | 32.0%      |

### 📄 `app.js`

**Path:** `src/app.js`
**Size:** 3.2 KB
**Language:** javascript (high confidence)
**Category:** Web Frontend

```javascript
// Your code here with proper syntax highlighting
```

📝 Text Format (Legacy)

The classic plain text format with directory tree and file contents:

git-ingest --format text

Example:

Directory structure for: /path/to/project
Generated on: 2024-01-15T10:30:00.000Z
Total items: 15

├── package.json
├── README.md
├── src/
│   ├── index.js
│   └── utils/
│       └── helpers.js

================================================
File: package.json
================================================
{
  "name": "my-project",
  "version": "1.0.0"
}

🔧 Development

Prerequisites

• Node.js 16+
• npm or yarn

Setup

# Clone the repository
git clone https://github.com/AungMyoKyaw/git-ingest.git
cd git-ingest

# Install dependencies
npm install

# Run tests
npm test

# Lint code
npm run lint

# Format code
npm run format

Testing

# Run all tests
npm test

# Run tests with coverage
npm run test:coverage

# Run tests in watch mode
npm run test:watch

Building

# Lint and format
npm run lint:fix
npm run format

# Test before release
npm test

🛠️ Configuration

Git-Ingest respects standard .gitignore files and includes sensible defaults:

Default Exclusions

• Generated output files (git-ingest-*.txt, git-ingest-*.json)
• Version control (.git/)
• Dependencies (node_modules/, vendor/)
• Build directories (dist/, build/, out/)
• Binary files (images, videos, executables)
• IDE files (.vscode/, .idea/)
• OS files (.DS_Store, Thumbs.db)

Custom Configuration

Use include/exclude patterns to customize file selection:

# Only include JavaScript and TypeScript files
git-ingest --include "*.js" "*.ts" "*.jsx" "*.tsx"

# Exclude test files
git-ingest --exclude "*.test.js" "*.spec.js" "__tests__/"

# Complex filtering
git-ingest --include "src/**" --exclude "*.min.js" "*.bundle.js"

🔍 Use Cases

📝 Code Reviews

Generate comprehensive project overviews for code reviews, sharing complete project context in a single document.

🤖 AI Analysis

Perfect for feeding entire codebases to AI tools like ChatGPT, Claude, or GitHub Copilot for analysis, documentation, or refactoring suggestions.

📚 Documentation

Create snapshot documentation of project structure and content for onboarding or archival purposes.

🔄 Project Migration

Analyze and understand legacy codebases before migration or modernization efforts.

👥 Team Collaboration

Share project context quickly with team members or stakeholders.

🚀 Performance

Git-Ingest v2.0 delivers significant performance improvements:

• 10x faster than v1.0 for large projects
• Constant memory usage regardless of project size
• Streaming architecture prevents memory overflow
• Async operations ensure UI responsiveness
• Smart caching reduces redundant operations

Benchmarks

Project Size	Files	Processing Time	Memory Usage
Small	< 100	< 2 seconds	< 50MB
Medium	1,000	< 10 seconds	< 100MB
Large	10,000	< 60 seconds	< 200MB

🔐 Security

Security is a top priority in Git-Ingest v2.0:

• No Command Injection: Safe clipboard operations
• Path Validation: Prevents directory traversal attacks
• Permission Respect: Works within user permissions
• Input Sanitization: All user inputs are validated
• No Data Persistence: Sensitive data is not cached

🆚 Version 2.0 Improvements

Git-Ingest v2.0 represents a complete rewrite with focus on security, performance, and user experience:

🔒 Security Enhancements

• ✅ Eliminated command injection vulnerabilities
• ✅ Added comprehensive input validation
• ✅ Implemented secure clipboard operations
• ✅ Added path traversal protection

⚡ Performance Optimizations

• ✅ Converted to fully async operations
• ✅ Implemented memory-efficient streaming
• ✅ Added binary file detection
• ✅ Optimized directory traversal

🎯 User Experience Improvements

• ✅ Professional CLI with Commander.js
• ✅ Real-time progress indicators
• ✅ Colored output and better formatting
• ✅ Enhanced error messages
• ✅ Comprehensive help system

🛠️ Developer Experience

• ✅ Complete test suite with Jest
• ✅ ESLint and Prettier configuration
• ✅ Modular, maintainable code structure
• ✅ Comprehensive documentation

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details on how to get started.

Quick Start for Contributors

• Fork the repository
• Create a feature branch: git checkout -b feature/amazing-feature
• Make your changes
• Run tests: npm test
• Lint code: npm run lint:fix
• Commit changes: git commit -m 'Add amazing feature'
• Push to branch: git push origin feature/amazing-feature
• Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Commander.js for CLI argument parsing
• Chalk for terminal colors
• Ora for progress indicators
• ignore for gitignore support
• isbinaryfile for binary detection
• clipboardy for clipboard operations

Made with ❤️ by Aung Myo Kyaw

⭐ If you find Git-Ingest useful, please star the repository!