@gitnaseem745/xtract

Xtract 📄➡️📊

A powerful PDF-to-Excel extraction tool designed specifically for converting Kruti Dev Unicode PDF documents (voter lists, family registers, etc.) into structured Excel spreadsheets.

✨ Features

• PDF to Excel Conversion — Extract tabular data from PDF files into clean Excel format
• Two Extraction Modes — raw for direct extraction, final for formatted output
• Post-Processing Options — Automatically populate birthplace, mobile numbers, and family data
• CLI & Programmatic API — Use from command line or integrate into your Node.js projects
• Debug Mode — Troubleshoot extractions with detailed logging

🎯 Use Cases

Xtract is designed to handle a wide variety of PDF-to-Excel conversion scenarios, particularly for Indian government documents and regional language content.

🗳️ Electoral & Government Records

Use Case	Description
Voter Lists	Convert PDF voter rolls to searchable Excel databases
Electoral Rolls	Digitize constituency-wise voter data for analysis
BPL/APL Lists	Process Below/Above Poverty Line beneficiary lists
Ration Card Data	Extract ration card holder information
Pension Records	Convert pension beneficiary PDFs to spreadsheets

👨‍👩‍👧‍👦 Family & Census Data

Use Case	Description
Family Registers	Extract family-wise demographic data
Parivar Registers	Digitize traditional family record books
Census Data	Process population census PDF reports
Household Surveys	Convert survey PDFs to analyzable data

🏛️ Panchayat & Local Bodies

Use Case	Description
Gram Panchayat Records	Village-level administrative data
Ward-wise Lists	Municipal ward population data
Property Tax Records	Extract property owner information
Birth/Death Registers	Vital statistics digitization

🔬 Research & Analysis

Use Case	Description
Demographic Studies	Population analysis from government PDFs
Migration Patterns	Track family movement across regions
Social Research	Extract data for academic studies
Policy Analysis	Government scheme beneficiary analysis

🏢 Organizations & NGOs

Use Case	Description
Beneficiary Tracking	Track scheme beneficiaries
Community Mapping	Map family relationships in communities
Outreach Planning	Plan health/education outreach programs
Donation Management	Process donor/recipient lists

📊 Business Applications

Use Case	Description
Customer Data Entry	Bulk convert customer PDFs to CRM-ready Excel
Survey Processing	Field survey data digitization
Report Conversion	Convert legacy PDF reports to editable format
Data Migration	Migrate PDF archives to database systems

💡 Tip: Xtract works best with PDFs containing Kruti Dev Unicode text. For scanned documents, use OCR software first to convert images to text-based PDFs.

📦 Installation

Global Installation (Recommended for CLI)

npm install -g xtract

Local Installation (For programmatic use)

npm install xtract

From Source

git clone <repository-url>
cd xtract
npm install
npm link  # Makes 'xtract' command available globally

🚀 Quick Start

Basic Usage

# Convert a PDF to Excel (final formatted output)
xtract -i families.pdf -o families.xlsx

# Extract raw data without formatting
xtract -i families.pdf -o raw-output.xlsx -m raw

📖 CLI Reference

xtract --input <pdf> --output <xlsx> [options]

Required Options

Option	Short	Description
--input	-i	Path to input PDF file

Optional Options

Option	Short	Default	Description
--output	-o	output.xlsx	Path to output Excel file
--mode	-m	final	Extraction mode: raw or final
--debug	—	false	Enable debug output for troubleshooting
--birthplace	—	—	Value to populate in birthplace column
--mobile	—	—	Extract and populate mobile numbers
--population	—	—	Populate family data (relations, count)
--populate	—	—	Comma-separated list: birthplace,mobile,population

Extraction Modes Explained

Mode	Description	Use Case
raw	Direct PDF text extraction to Excel	When you need unprocessed data for custom processing
final	Formatted output with structured columns	Ready-to-use voter list / family register format

📋 Examples

1. Basic Conversion

Convert a PDF to a formatted Excel file:

xtract -i voter-list.pdf -o voter-list.xlsx

2. Raw Extraction

Extract raw data without any formatting:

xtract -i document.pdf -o raw-data.xlsx --mode raw

3. With Birthplace Population

Populate a specific birthplace value in all records:

xtract -i families.pdf -o output.xlsx --birthplace "पीपली"

4. Full Processing Pipeline

Extract with all post-processing options:

xtract -i families.pdf -o complete.xlsx \
  --birthplace "मो हयातपुर" \
  --mobile \
  --population

5. Debug Mode

Troubleshoot extraction issues:

xtract -i problem-file.pdf -o output.xlsx --debug

6. Using --populate Flag

Combine multiple post-processing options:

xtract -i families.pdf -o out.xlsx --populate birthplace,mobile --birthplace "गाँव"

💻 Programmatic API

Import the Module

import { runPipeline, convertPdfToExcel, convertPdfToFinal } from 'xtract';

High-Level Pipeline

The easiest way to convert PDFs:

await runPipeline({
  inputPdf: 'input.pdf',
  outputExcel: 'output.xlsx',
  mode: 'final',           // 'raw' | 'final'
  options: { debug: true }
});

Low-Level Functions

For more control over the conversion process:

// Step 1: Convert PDF to raw Excel
await convertPdfToExcel('input.pdf', 'raw.xlsx');

// Step 2: Convert raw Excel to final formatted output
await convertPdfToFinal('raw.xlsx', 'final.xlsx', {
  birthPlace: 'पीपली'
});

📚 API Reference

runPipeline(params)

Orchestrates the full PDF-to-Excel conversion pipeline.

Parameters:

Parameter	Type	Required	Default	Description
inputPdf	string	✅	—	Path to input PDF file
outputExcel	string	❌	output.xlsx	Path to output Excel file
mode	string	❌	final	Extraction mode: raw or final
options	object	❌	{}	Additional options (e.g., { debug: true })

Returns: Promise<void>

Example:

await runPipeline({
  inputPdf: 'families.pdf',
  outputExcel: 'families.xlsx',
  mode: 'final',
  options: { debug: false }
});

convertPdfToExcel(inputPdf, outputExcel, options)

Converts PDF directly to Excel format (raw extraction).

Parameters:

Parameter	Type	Required	Description
inputPdf	string	✅	Path to input PDF
outputExcel	string	✅	Path to output Excel
options	object	❌	Options like { debug: true }

Returns: Promise<void>

convertPdfToFinal(rawExcel, outputExcel, options)

Converts raw Excel to final formatted output.

Parameters:

Parameter	Type	Required	Description
rawExcel	string	✅	Path to raw Excel file
outputExcel	string	✅	Path to final output Excel
options	object	❌	Options like { birthPlace: 'value' }

Returns: Promise<void>

🔧 Requirements

• Node.js >= 18.0.0
• npm >= 8.0.0

📁 Project Structure

xtract/
├── bin/
│   └── xtract.js          # CLI entry point
├── src/
│   ├── index.js           # Public API exports
│   ├── pipeline/          # Core conversion logic
│   ├── extractors/        # PDF text extractors
│   ├── utils/             # Helper utilities
│   └── config/            # Configuration
├── package.json
└── README.md

🐛 Troubleshooting

Common Issues

1. "Cannot find module" error

npm install  # Reinstall dependencies

2. Empty output file

• Ensure your PDF contains extractable text (not scanned images)
• Try --debug flag to see extraction details

3. Garbled text in output

• The tool is optimized for Kruti Dev Unicode. Other fonts may not work correctly.

4. Permission denied

chmod +x bin/xtract.js  # Make CLI executable

📄 License

MIT © 2024

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.