Lightning-fast file type detection using magic bytes (file signatures) with a focus on stream processing and minimal memory usage
Lightning-fast file type detection using magic bytes (file signatures) with a focus on stream processing and minimal memory usage.
Features • Installation • Quick Start • API • File Types • Performance
npm install mime-bytes
import { FileTypeDetector } from 'mime-bytes';
import { createReadStream } from 'fs';
const detector = new FileTypeDetector();
// Stream-based detection (recommended)
const stream = createReadStream('document.pdf');
const fileType = await detector.detectFromStream(stream);
console.log(fileType);
// {
// name: "pdf",
// mimeType: "application/pdf",
// extensions: ["pdf"],
// description: "Portable Document Format",
// charset: "binary",
// contentType: "application/pdf",
// confidence: 1.0
// }
The main class for file type detection.
const detector = new FileTypeDetector({
peekBytes: 32, // Number of bytes to peek (default: 32)
checkMultipleOffsets: true, // Check offsets 0, 4, 8, 12 (default: true)
maxOffset: 12 // Maximum offset to check (default: 12)
});
detectFromStream(stream: Readable): Promise<FileTypeResult | null>Detect file type from a readable stream. This is the primary and recommended method.
const stream = createReadStream('video.mp4');
const result = await detector.detectFromStream(stream);
// Stream can still be used after detection!
detectFromBuffer(buffer: Buffer): Promise<FileTypeResult | null>Detect file type from a buffer (for already-loaded data).
const buffer = await fs.readFile('image.png');
const result = await detector.detectFromBuffer(buffer);
detectWithFallback(input: Readable | Buffer, filename?: string): Promise<FileTypeResult | null>Detect with automatic fallback to extension-based detection.
const stream = createReadStream('document.docx');
const result = await detector.detectWithFallback(stream, 'document.docx');
// Will use magic bytes first, then fall back to extension if needed
detectFromExtension(extension: string): FileTypeResult[]Get possible file types based on extension alone.
const results = detector.detectFromExtension('.jpg');
// Returns array of possible types with lower confidence scores
addFileType(fileType: FileTypeDefinition): voidAdd a custom file type definition.
detector.addFileType({
name: "myformat",
magicBytes: ["0x4D", "0x59", "0x46", "0x4D"],
mimeType: "application/x-myformat",
extensions: ["myf", "myfmt"],
description: "My Custom Format",
category: "application"
});
removeFileType(name: string): booleanRemove a file type by name.
detector.removeFileType('myformat'); // Returns true if removed
getByCategory(category: string): FileTypeDefinition[]Get all file types in a specific category.
const imageTypes = detector.getByCategory('image');
const audioTypes = detector.getByCategory('audio');
isFileType(input: Buffer | FileTypeResult, typeName: string): booleanCheck if a buffer or result matches a specific file type.
const buffer = await fs.readFile('image.png');
if (detector.isFileType(buffer, 'png')) {
console.log('This is a PNG file!');
}
getStats(): FileTypeStatsGet detection statistics.
const stats = detector.getStats();
console.log(`Total detections: ${stats.totalDetections}`);
console.log(`Cache hit rate: ${stats.cacheHitRate}%`);
clearCache(): voidClear the internal cache (useful for testing or memory management).
detector.clearCache();
All detection methods return a FileTypeResult object:
interface FileTypeResult {
name: string; // Short identifier (e.g., "pdf")
mimeType: string; // Standard MIME type
extensions: string[]; // Common file extensions
description?: string; // Human-readable description
charset?: string; // Character encoding (for text files)
contentType?: string; // Full content type
confidence: number; // Detection confidence (0-1)
}
mime-bytes is designed for speed and efficiency:
// First detection: ~13ms
// Cached detection: ~8ms (38% faster)
// Concurrent processing: Handles 1000+ files/second
For files with magic bytes at unusual offsets:
const detector = new FileTypeDetector({
peekBytes: 64, // Read more bytes
maxOffset: 32 // Check deeper offsets
});
import { pipeline } from 'stream/promises';
async function processLargeFile(filepath: string) {
const readStream = createReadStream(filepath);
// Detect type without consuming the stream
const fileType = await detector.detectFromStream(readStream);
if (fileType?.name === 'zip') {
// Continue processing the same stream
await pipeline(
readStream,
createUnzipStream(),
createWriteStream('output')
);
}
}
// TypeScript files can be video/mp2t or text/x-typescript
const result = await detector.detectWithFallback(stream, 'file.ts');
if (result?.charset === 'utf-8') {
console.log('TypeScript source file');
} else if (result?.charset === 'binary') {
console.log('MPEG Transport Stream');
}
async function detectMultipleFiles(files: string[]) {
const results = await Promise.all(
files.map(async (file) => {
const stream = createReadStream(file);
const type = await detector.detectFromStream(stream);
return { file, type };
})
);
return results;
}
try {
const result = await detector.detectFromStream(stream);
if (!result) {
console.log('Unknown file type');
// Handle unknown format
} else {
console.log(`Detected: ${result.name}`);
}
} catch (error) {
console.error('Detection failed:', error.message);
// Handle stream errors, permission issues, etc.
}
Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.
src/file-types-registry.tsSET LOCAL) into queries—ideal for setting role, jwt.claims, and other session settings.libpg_query, converting SQL into parse trees.SELECT, INSERT, UPDATE, DELETE, and stored procedure calls—supports advanced SQL features like JOIN, GROUP BY, and schema-qualified queries.AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED "AS IS", AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND.
No developer or entity involved in creating this software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the code, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value.
FAQs
Lightning-fast file type detection using magic bytes (file signatures) with a focus on stream processing and minimal memory usage
The npm package mime-bytes receives a total of 65 weekly downloads. As such, mime-bytes popularity was classified as not popular.
We found that mime-bytes demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
npm now supports Trusted Publishing with OIDC, enabling secure package publishing directly from CI/CD workflows without relying on long-lived tokens.
Research
/Security News
A RubyGems malware campaign used 60 malicious packages posing as automation tools to steal credentials from social media and marketing tool users.
Security News
The CNA Scorecard ranks CVE issuers by data completeness, revealing major gaps in patch info and software identifiers across thousands of vulnerabilities.