toon-formatter
Advanced tools
A powerful library to convert between TOON, JSON, YAML, XML, and CSV with end-to-end encryption. Includes Unified Converters for direct format-to-format translation and LLM token optimization.
A lightweight library to convert between TOON (Token-Oriented Object Notation) and popular data formats (JSON, YAML, XML, CSV).
Reduce your LLM token costs by up to 40% using the TOON format!
* Only external dependencies: js-yaml and papaparse for YAML and CSV parsing
npm install toon-formatter
TOON (Token-Oriented Object Notation) is a compact, human-readable data serialization format designed specifically for use with Large Language Models (LLMs). It represents the same data model as JSON but with significantly fewer tokens.
JSON (87 tokens):
{
"users": [
{"id": 1, "name": "Alice", "active": true},
{"id": 2, "name": "Bob", "active": false},
{"id": 3, "name": "Charlie", "active": true}
]
}
TOON (52 tokens - 40% reduction):
users[3]{id,name,active}:
1,"Alice",true
2,"Bob",false
3,"Charlie",true
NEW in v2.2.0: TOON Formatter now includes a powerful CLI utility for fast data conversion directly from your terminal!
To use the toon-formatter command anywhere:
npm install -g toon-formatter
Or run it instantly without installation using npx:
npx toon-formatter --help
Convert files or piped data easily:
# Convert JSON file to TOON
toon-formatter --from json --to toon -i data.json -o data.toon
# Piping data (JSON -> TOON)
echo '{"name": "Alice"}' | toon-formatter --from json --to toon
# Convert XML to JSON (prettified by default)
cat profile.xml | toon-formatter --from xml --to json
The CLI supports all library features, including validation and encryption:
# Validate a TOON string
cat data.toon | toon-formatter --validate toon
# Encrypt data during conversion (XOR)
echo '{"secret": "data"}' | toon-formatter --from json --to toon --mode export --key "mykey" --algo xor
# Decrypt data (AES-256-GCM)
cat encrypted.txt | toon-formatter --from toon --to json --mode ingestion --key "your-32-byte-key" --algo aes-256-gcm
| Flag | Short | Description |
|---|---|---|
--from | -f | Input format (json, yaml, xml, csv, toon) |
--to | -t | Output format (json, yaml, xml, csv, toon) |
--input | -i | Input file path (defaults to stdin) |
--output | -o | Output file path (defaults to stdout) |
--validate | -v | Validate the input format and exit |
--mode | -m | Encryption mode (middleware, ingestion, export) |
--key | -k | Encryption key |
--algo | -a | Encryption algorithm (aes-256-gcm, xor, base64) |
--async | Use asynchronous conversion mode | |
--no-parse | Skip parsing of objects (returns raw strings) | |
--help | -h | Show help information |
import { jsonToToonSync, toonToJsonSync } from 'toon-formatter';
// JSON to TOON
const jsonData = { name: "Alice", age: 30, active: true };
const toonString = jsonToToonSync(jsonData);
console.log(toonString);
// Output:
// name: "Alice"
// age: 30
// active: true
// TOON to JSON
const toonInput = `name: "Alice"\nage: 30\nactive: true`;
const jsonOutput = toonToJsonSync(toonInput);
console.log(jsonOutput);
// Output: { name: "Alice", age: 30, active: true }
import { jsonToToon, toonToJson } from 'toon-formatter';
// JSON to TOON (async)
const jsonData = { name: "Alice", age: 30, active: true };
const toonString = await jsonToToon(jsonData);
console.log(toonString);
// TOON to JSON (async)
const toonInput = `name: "Alice"\nage: 30\nactive: true`;
const jsonOutput = await toonToJson(toonInput);
console.log(jsonOutput);
JSON, XML, and CSV to TOON conversions support both full data strings AND mixed text with embedded data!
import { jsonToToonSync, xmlToToon, csvToToonSync } from 'toon-formatter';
// Example 1: Extract and convert JSON from mixed text
const mixedText = `
Here's some user data:
{"name": "Alice", "age": 30, "role": "Engineer"}
And here's another object:
{"name": "Bob", "age": 25, "role": "Designer"}
`;
const result = jsonToToonSync(mixedText);
console.log(result);
// Output:
// Here's some user data:
// name: "Alice"
// age: 30
// role: "Engineer"
//
// And here's another object:
// name: "Bob"
// age: 25
// role: "Designer"
// Example 2: Extract and convert XML from mixed text
const xmlMixedText = `
The user profile is:
<user><name>Alice</name><age>30</age></user>
`;
const xmlResult = await xmlToToon(xmlMixedText);
console.log(xmlResult);
// Output:
// The user profile is:
// user:
// name: "Alice"
// age: "30"
// Example 3: Extract and convert CSV from mixed text
const csvMixedText = `
Employee data:
name,role,salary
Alice,Engineer,100000
Bob,Designer,95000
`;
const csvResult = csvToToonSync(csvMixedText);
// Converts the CSV table to TOON format while preserving surrounding text
Important Notes:
jsonToToon, xmlToToon, csvToToonyamlToToon, toonToJson, toonToXml, toonToCsv, toonToYamlimport ToonConverter from 'toon-formatter';
// Synchronous conversions (default methods)
const toonFromJson = ToonConverter.fromJson({ key: "value" });
const toonFromYaml = ToonConverter.fromYaml("key: value");
const toonFromXml = ToonConverter.fromXml("<root><key>value</key></root>");
const toonFromCsv = ToonConverter.fromCsv("name,age\nAlice,30");
// Asynchronous conversions (methods with 'Async' suffix)
const toonFromJsonAsync = await ToonConverter.fromJsonAsync({ key: "value" });
const toonFromYamlAsync = await ToonConverter.fromYamlAsync("key: value");
const toonFromXmlAsync = await ToonConverter.fromXmlAsync("<root><key>value</key></root>");
const toonFromCsvAsync = await ToonConverter.fromCsvAsync("name,age\nAlice,30");
// Convert to various formats (synchronous by default)
const jsonData = ToonConverter.toJson(toonString);
const yamlData = ToonConverter.toYaml(toonString);
const xmlData = ToonConverter.toXml(toonString);
const csvData = ToonConverter.toCsv(toonString);
// Asynchronous versions (methods with 'Async' suffix)
const jsonDataAsync = await ToonConverter.toJsonAsync(toonString);
const yamlDataAsync = await ToonConverter.toYamlAsync(toonString);
const xmlDataAsync = await ToonConverter.toXmlAsync(toonString);
const csvDataAsync = await ToonConverter.toCsvAsync(toonString);
// Validate TOON (synchronous by default)
const result = ToonConverter.validate(toonString);
if (result.isValid) {
console.log("Valid TOON!");
} else {
console.error("Invalid TOON:", result.error);
}
// Validate TOON (asynchronous)
const resultAsync = await ToonConverter.validateAsync(toonString);
All conversion functions are available in both synchronous and asynchronous versions:
Synchronous Functions (Suffix: Sync)
jsonToToonSync(), toonToJsonSync()yamlToToonSync(), toonToYamlSync()xmlToToonSync(), toonToXmlSync()csvToToonSync(), toonToCsvSync()validateToonStringSync()Use when: You need immediate results and are working in a synchronous context.
Asynchronous Functions (No suffix)
jsonToToon(), toonToJson()yamlToToon(), toonToYaml()xmlToToon(), toonToXml()csvToToon(), toonToCsv()validateToonString()Use when: You're in an async context or want to maintain consistency with async/await patterns.
Synchronous Methods (No suffix - default)
ToonConverter.fromJson(), ToonConverter.toJson()ToonConverter.fromYaml(), ToonConverter.toYaml()ToonConverter.fromXml(), ToonConverter.toXml()ToonConverter.fromCsv(), ToonConverter.toCsv()ToonConverter.validate()Asynchronous Methods (Suffix: Async)
ToonConverter.fromJsonAsync(), ToonConverter.toJsonAsync()ToonConverter.fromYamlAsync(), ToonConverter.toYamlAsync()ToonConverter.fromXmlAsync(), ToonConverter.toXmlAsync()ToonConverter.fromCsvAsync(), ToonConverter.toCsvAsync()ToonConverter.validateAsync()Note:
Sync suffix, async functions have no suffixAsync suffixxmldom package if neededNEW in v2.3.0: The TOON Converter now features Smart Code Optimization, a preprocessing pipeline designed to maximize token efficiency when dealing with mixed text and code blocks.
LLMs often process documentation or chat history that contains a mix of natural language, code snippets, and data (JSON/XML/CSV). Smart Code Optimization automatically cleans and compresses this content before conversion, resulting in even lower token counts.
The library uses heuristics to detect code blocks (npm/git commands, shebangs, common programming patterns). Detected blocks are:
# or //) are removed.The library contains a dictionary of 100+ verbose phrases (Contracted/Common/Technical) and automatically replaces them with token-efficient abbreviations.
[!IMPORTANT] To maintain syntactic integrity, these replacements are ONLY applied to natural language text. Both code blocks and data blocks (JSON/XML/CSV/TOON) are strictly preserved and never modified by this process.
When using jsonToToon, xmlToToon, or csvToToon, the library:
Input Mixed Text:
Please review this large language model configuration as soon as possible:
{"model": "gpt-4", "temp": 0.7}
npm install openai // install the helper library
Optimized TOON Output:
pls review this llm configuration asap:
model: "gpt-4"
temp: 0.7
npm install openai
Smart Code Optimization is active by default for all methods starting with jsonTo, xmlTo, and csvTo:
jsonToToon(), jsonToYaml(), jsonToXml()...xmlToToon(), xmlToJson(), xmlToCsv()...csvToToon(), csvToJson(), csvToXml()...yamlToToon(), toonToJson() (These remain Pure Data only)Mixed text support allows you to convert data that's embedded within regular text, not just pure data strings. This is incredibly useful for processing documentation, API responses, or any content that contains data snippets.
| Conversion | Full Data | Mixed Text | Notes |
|---|---|---|---|
jsonToToon() | ✅ | ✅ | Extracts all JSON objects/arrays |
xmlToToon() | ✅ | ✅ | Extracts all XML elements |
csvToToon() | ✅ | ✅ | Extracts CSV tables |
yamlToToon() | ✅ | ❌ | Pure YAML only |
toonToJson() | ✅ | ❌ | Pure TOON only |
toonToXml() | ✅ | ❌ | Pure TOON only |
toonToCsv() | ✅ | ❌ | Pure TOON only |
toonToYaml() | ✅ | ❌ | Pure TOON only |
import { jsonToToonSync } from 'toon-formatter';
const documentation = `
# API Documentation
## User Endpoint
Returns: {"id": 1, "name": "Alice", "role": "admin"}
## Product Endpoint
Returns: {"id": 101, "title": "Widget", "price": 29.99}
`;
const converted = jsonToToonSync(documentation);
console.log(converted);
// Output:
// # API Documentation
//
// ## User Endpoint
// Returns: id: 1
// name: "Alice"
// role: "admin"
//
// ## Product Endpoint
// Returns: id: 101
// title: "Widget"
// price: 29.99
---
## 🔐 Encryption Support
**NEW in v2.0.0**: The TOON Converter now supports end-to-end encryption for secure data transmission and storage!
### Overview
The encryption feature allows you to:
- **Encrypt data before transmission** to protect sensitive information
- **Store encrypted TOON data** securely
- **Process encrypted data** without exposing plaintext
- **Support multiple encryption algorithms**: AES-256-GCM, XOR, Base64
### Quick Start with Encryption
```javascript
import { ToonConverter, Encryptor } from 'toon-formatter';
// 1. Generate a secure encryption key
const key = Encryptor.generateKey(); // 32-byte key for AES-256-GCM
// 2. Create an encryptor
const encryptor = new Encryptor(key, 'aes-256-gcm');
// 3. Create a converter with encryption support
const converter = new ToonConverter(encryptor);
// 4. Convert and encrypt data
const data = { user: "Alice", role: "admin" };
const encryptedToon = converter.fromJson(data, {
conversionMode: 'export'
});
console.log(encryptedToon); // Encrypted string
// 5. Decrypt and convert back
const decrypted = encryptor.decrypt(encryptedToon);
console.log(decrypted); // Plain TOON string
High-security authenticated encryption with Galois/Counter Mode.
const key = Encryptor.generateKey(); // Generates 32-byte key
const encryptor = new Encryptor(key, 'aes-256-gcm');
Features:
Simple obfuscation (not cryptographically secure).
const encryptor = new Encryptor('my-secret-key', 'xor');
Use cases:
Simple encoding (not encryption).
const encryptor = new Encryptor(null, 'base64');
Use cases:
The encryption system supports 4 conversion modes for different data flow scenarios:
no_encryption (Default)No encryption applied - standard conversion.
const converter = new ToonConverter(encryptor);
const toon = converter.fromJson(data); // Plain TOON
middleware ModeEncrypted → Encrypted (Decrypt → Convert → Re-encrypt)
Perfect for middleware services that need to convert format without exposing data.
// Input: Encrypted JSON
const encryptedJson = '...'; // From client
// Convert to encrypted TOON (never see plaintext)
const encryptedToon = converter.fromJson(encryptedJson, {
conversionMode: 'middleware'
});
// Output: Encrypted TOON (can be stored or forwarded)
Use case: API gateway converting encrypted client data to encrypted storage format.
ingestion ModeEncrypted → Plain (Decrypt → Convert)
For ingesting encrypted data into your system.
// Input: Encrypted JSON from external source
const encryptedJson = '...';
// Convert to plain TOON for processing
const plainToon = converter.fromJson(encryptedJson, {
conversionMode: 'ingestion'
});
// Output: Plain TOON (ready for processing)
Use case: Receiving encrypted data from clients and converting to internal format.
export ModePlain → Encrypted (Convert → Encrypt)
For exporting data securely.
// Input: Plain JSON data
const data = { user: "Alice", role: "admin" };
// Convert and encrypt for transmission
const encryptedToon = converter.fromJson(data, {
conversionMode: 'export'
});
// Output: Encrypted TOON (safe to transmit)
Use case: Sending data to external systems or clients securely.
import { ToonConverter, Encryptor } from 'toon-formatter';
// Setup (same key on client and server)
const key = Encryptor.generateKey();
const encryptor = new Encryptor(key, 'aes-256-gcm');
// CLIENT SIDE
// ============
const clientConverter = new ToonConverter(encryptor);
// 1. User submits sensitive data
const userData = {
ssn: "123-45-6789",
creditCard: "4111-1111-1111-1111",
email: "alice@example.com"
};
// 2. Encrypt before sending
const encryptedPayload = encryptor.encrypt(JSON.stringify(userData));
// 3. Send to server
await fetch('/api/user', {
method: 'POST',
body: encryptedPayload
});
// SERVER SIDE (Middleware)
// =========================
const serverConverter = new ToonConverter(encryptor);
// 4. Receive encrypted data
const encryptedJson = await request.text();
// 5. Convert to encrypted TOON for storage (middleware mode)
const encryptedToon = serverConverter.fromJson(encryptedJson, {
conversionMode: 'middleware'
});
// 6. Store encrypted TOON in database
await db.save(encryptedToon);
// SERVER SIDE (Processing)
// =========================
// 7. Retrieve encrypted TOON
const storedToon = await db.get(userId);
// 8. Convert back to plain JSON for processing (ingestion mode)
const plainData = serverConverter.toJson(storedToon, {
conversionMode: 'ingestion',
returnJson: true
});
// 9. Process data
const user = JSON.parse(plainData);
console.log(user.email); // alice@example.com
returnJson ParameterBy default, toJson() returns a JavaScript object. For encryption modes, you need a string. Use returnJson: true:
// Returns object (default)
const obj = converter.toJson(toonString);
console.log(obj); // { name: "Alice" }
// Returns JSON string (for encryption)
const jsonString = converter.toJson(toonString, { returnJson: true });
console.log(jsonString); // '{"name":"Alice"}'
// With encryption
const encrypted = converter.toJson(toonString, {
conversionMode: 'export',
returnJson: true // Required for encryption!
});
// Generate a secure random key
const key = Encryptor.generateKey();
// Store as Base64 (e.g., in environment variables)
const keyBase64 = key.toString('base64');
process.env.ENCRYPTION_KEY = keyBase64;
// Load from storage
const loadedKey = Buffer.from(process.env.ENCRYPTION_KEY, 'base64');
const encryptor = new Encryptor(loadedKey, 'aes-256-gcm');
// Old system
const oldKey = Buffer.from(process.env.OLD_KEY, 'base64');
const oldEncryptor = new Encryptor(oldKey, 'aes-256-gcm');
// New system
const newKey = Encryptor.generateKey();
const newEncryptor = new Encryptor(newKey, 'aes-256-gcm');
// Migrate data
const encryptedData = await db.getAllEncrypted();
for (const item of encryptedData) {
// Decrypt with old key
const plaintext = oldEncryptor.decrypt(item.data);
// Re-encrypt with new key
const reEncrypted = newEncryptor.encrypt(plaintext);
// Update database
await db.update(item.id, reEncrypted);
}
// Update environment variable
process.env.ENCRYPTION_KEY = newKey.toString('base64');
try {
const encrypted = encryptor.encrypt(data);
const decrypted = encryptor.decrypt(encrypted);
} catch (error) {
if (error.message.includes('decryption failed')) {
console.error('Wrong key or tampered data');
} else if (error.message.includes('Invalid encrypted data format')) {
console.error('Corrupted ciphertext');
} else {
console.error('Encryption error:', error.message);
}
}
All encryption operations work with async methods:
const converter = new ToonConverter(encryptor);
// Async conversion with encryption
const encrypted = await converter.fromJsonAsync(data, {
conversionMode: 'export'
});
const decrypted = await converter.toJsonAsync(encrypted, {
conversionMode: 'ingestion',
returnJson: true
});
Before (no encryption):
import { ToonConverter } from 'toon-formatter';
const toon = ToonConverter.fromJson(data);
const json = ToonConverter.toJson(toon);
After (with encryption):
import { ToonConverter, Encryptor } from 'toon-formatter';
// Create encryptor
const key = Encryptor.generateKey();
const encryptor = new Encryptor(key, 'aes-256-gcm');
// Create converter instance
const converter = new ToonConverter(encryptor);
// Use instance methods
const encrypted = converter.fromJson(data, { conversionMode: 'export' });
const plain = converter.toJson(encrypted, { conversionMode: 'ingestion' });
Note: Static methods still work for backward compatibility (no encryption).
For high-throughput applications, consider:
jsonToToonSync(data, key?, depth?)Converts JSON data to TOON format (synchronous).
Supports: ✅ Full JSON data, ✅ Mixed text with embedded JSON
Parameters:
data (any): JSON data to convert, or string containing JSONkey (string, optional): Key name for root objectdepth (number, optional): Initial indentation depthReturns: string - TOON formatted string
Example:
import { jsonToToonSync } from 'toon-formatter';
// Full JSON data
const data = {
users: [
{ id: 1, name: "Alice" },
{ id: 2, name: "Bob" }
]
};
const toon = jsonToToonSync(data);
// Mixed text with embedded JSON
const mixedText = 'User: {"name": "Alice", "age": 30}';
const result = jsonToToonSync(mixedText);
// Output: User: name: "Alice"\nage: 30
jsonToToon(data)Converts JSON data to TOON format (asynchronous).
Supports: ✅ Full JSON data, ✅ Mixed text with embedded JSON
Parameters:
data (any): JSON data to convert, or string containing JSONReturns: Promise<string> - TOON formatted string
toonToJsonSync(toonString, returnJson?)Converts TOON string to JSON (synchronous).
Supports: ❌ Pure TOON data only (no mixed text)
Parameters:
toonString (string): TOON formatted stringreturnJson (boolean, optional): If true, returns JSON string; if false (default), returns objectReturns: any | string - Parsed JSON data (object by default, string if returnJson=true)
toonToJson(toonString, returnJson?)Converts TOON string to JSON (asynchronous).
Supports: ❌ Pure TOON data only (no mixed text)
Parameters:
toonString (string): TOON formatted stringreturnJson (boolean, optional): If true, returns JSON string; if false (default), returns objectReturns: Promise<any | string> - Parsed JSON data (object by default, string if returnJson=true)
yamlToToonSync(yamlString)Converts YAML to TOON format (synchronous).
Supports: ✅ Full YAML data, ✅ Mixed text with embedded YAML
Parameters:
yamlString (string): YAML formatted string or mixed textReturns: string - TOON formatted string
Throws: Error if YAML is invalid
yamlToToon(yamlString)Converts YAML to TOON format (asynchronous).
Supports: ✅ Full YAML data, ✅ Mixed text with embedded YAML
Parameters:
yamlString (string): YAML formatted string or mixed textReturns: Promise<string> - TOON formatted string
Throws: Error if YAML is invalid
toonToYamlSync(toonString)Converts TOON to YAML format (synchronous).
Supports: ❌ Pure TOON data only (no mixed text)
Parameters:
toonString (string): TOON formatted stringReturns: string - YAML formatted string
Throws: Error if TOON is invalid
toonToYaml(toonString)Converts TOON to YAML format (asynchronous).
Supports: ❌ Pure TOON data only (no mixed text)
Parameters:
toonString (string): TOON formatted stringReturns: Promise<string> - YAML formatted string
Throws: Error if TOON is invalid
xmlToToonSync(xmlString)Converts XML to TOON format (synchronous).
Supports: ✅ Full XML data, ✅ Mixed text with embedded XML
Parameters:
xmlString (string): XML formatted string or mixed textReturns: string - TOON formatted string
Throws: Error if XML is invalid
Note: Requires DOMParser (browser) or xmldom package (Node.js)
xmlToToon(xmlString)Converts XML to TOON format (asynchronous).
Supports: ✅ Full XML data, ✅ Mixed text with embedded XML
Parameters:
xmlString (string): XML formatted string or mixed textReturns: Promise<string> - TOON formatted string
Throws: Error if XML is invalid
Note: Automatically loads xmldom in Node.js environments
toonToXmlSync(toonString)Converts TOON to XML format (synchronous).
Supports: ❌ Pure TOON data only (no mixed text)
Parameters:
toonString (string): TOON formatted stringReturns: string - XML formatted string
Throws: Error if TOON is invalid
toonToXml(toonString)Converts TOON to XML format (asynchronous).
Supports: ❌ Pure TOON data only (no mixed text)
Parameters:
toonString (string): TOON formatted stringReturns: Promise<string> - XML formatted string
Throws: Error if TOON is invalid
csvToToonSync(csvString)Converts CSV to TOON format (synchronous).
Supports: ✅ Full CSV data, ✅ Mixed text with embedded CSV
Parameters:
csvString (string): CSV formatted string or mixed textReturns: string - TOON formatted string
Throws: Error if CSV is invalid
csvToToon(csvString)Converts CSV to TOON format (asynchronous).
Supports: ✅ Full CSV data, ✅ Mixed text with embedded CSV
Parameters:
csvString (string): CSV formatted string or mixed textReturns: Promise<string> - TOON formatted string
Throws: Error if CSV is invalid
toonToCsvSync(toonString)Converts TOON to CSV format (synchronous).
Supports: ❌ Pure TOON data only (no mixed text)
Parameters:
toonString (string): TOON formatted stringReturns: string - CSV formatted string
Throws: Error if TOON is invalid
toonToCsv(toonString)Converts TOON to CSV format (asynchronous).
Supports: ❌ Pure TOON data only (no mixed text)
Parameters:
toonString (string): TOON formatted stringReturns: Promise<string> - CSV formatted string
Throws: Error if TOON is invalid
validateToonStringSync(toonString)Validates a TOON string for syntax and structural correctness (synchronous).
Parameters:
toonString (string): TOON string to validateReturns: {isValid: boolean, error: string|null}
Example:
import { validateToonStringSync } from 'toon-formatter';
const result = validateToonStringSync(`
users[2]{id,name}:
1,"Alice"
2,"Bob"
`);
if (result.isValid) {
console.log("Valid TOON!");
} else {
console.error("Error:", result.error);
}
validateToonString(toonString)Validates a TOON string for syntax and structural correctness (asynchronous).
Parameters:
toonString (string): TOON string to validateReturns: Promise<{isValid: boolean, error: string|null}>
The Encryptor class provides encryption and decryption capabilities.
new Encryptor(key, algorithm)Creates a new Encryptor instance.
Parameters:
key (Buffer | string | null): Encryption key
aes-256-gcm: 32-byte Buffer (use Encryptor.generateKey())xor: String or Bufferbase64: null (no key needed)algorithm (string): Encryption algorithm - 'aes-256-gcm', 'xor', or 'base64'Example:
// AES-256-GCM (recommended)
const key = Encryptor.generateKey();
const encryptor = new Encryptor(key, 'aes-256-gcm');
// XOR
const xorEncryptor = new Encryptor('my-secret-key', 'xor');
// Base64
const base64Encryptor = new Encryptor(null, 'base64');
Encryptor.generateKey()Static method to generate a secure 32-byte encryption key for AES-256-GCM.
Returns: Buffer - 32-byte random key
Example:
const key = Encryptor.generateKey();
console.log(key.length); // 32
// Store as Base64
const keyBase64 = key.toString('base64');
process.env.ENCRYPTION_KEY = keyBase64;
// Load from Base64
const loadedKey = Buffer.from(process.env.ENCRYPTION_KEY, 'base64');
encryptor.encrypt(data)Encrypts a string.
Parameters:
data (string): Plaintext string to encryptReturns: string - Encrypted string (hex-encoded for AES-256-GCM and XOR, Base64 for base64)
Throws: Error if data is not a string or key is missing (for AES/XOR)
Example:
const encrypted = encryptor.encrypt('Hello, World!');
console.log(encrypted); // Hex string (AES-256-GCM)
encryptor.decrypt(encryptedData)Decrypts an encrypted string.
Parameters:
encryptedData (string): Encrypted stringReturns: string - Decrypted plaintext
Throws: Error if decryption fails, wrong key, or tampered data
Example:
const decrypted = encryptor.decrypt(encrypted);
console.log(decrypted); // 'Hello, World!'
The ToonConverter class now supports both static methods (backward compatible) and instance methods (with encryption).
new ToonConverter(encryptor?)Creates a new ToonConverter instance.
Parameters:
encryptor (Encryptor | null, optional): Encryptor instance for encryption supportExample:
// Without encryption
const converter = new ToonConverter();
// With encryption
const key = Encryptor.generateKey();
const encryptor = new Encryptor(key, 'aes-256-gcm');
const converter = new ToonConverter(encryptor);
All instance methods accept an options object with:
conversionMode (string): 'no_encryption' (default), 'middleware', 'ingestion', or 'export'returnJson (boolean, for toJson methods): If true, returns JSON string; if false (default), returns objectExample:
// fromJson with encryption
const encrypted = converter.fromJson(data, {
conversionMode: 'export'
});
// toJson with encryption and JSON string output
const jsonString = converter.toJson(toonString, {
conversionMode: 'ingestion',
returnJson: true
});
// fromYaml with middleware mode
const encryptedToon = converter.fromYaml(encryptedYaml, {
conversionMode: 'middleware'
});
Available instance methods:
fromJson(data, options?) / fromJsonAsync(data, options?)toJson(toonString, options?) / toJsonAsync(toonString, options?)fromYaml(yamlString, options?) / fromYamlAsync(yamlString, options?)toYaml(toonString, options?) / toYamlAsync(toonString, options?)fromXml(xmlString, options?) / fromXmlAsync(xmlString, options?)toXml(toonString, options?) / toXmlAsync(toonString, options?)fromCsv(csvString, options?) / fromCsvAsync(csvString, options?)toCsv(toonString, options?) / toCsvAsync(toonString, options?)validate(toonString) / validateAsync(toonString)Static methods work exactly as before, with no encryption support:
// Static usage (no encryption)
const toon = ToonConverter.fromJson(data);
const json = ToonConverter.toJson(toon);
// Static with returnJson parameter
const jsonString = ToonConverter.toJson(toon, true);
Note: For toJson and toJsonAsync static methods, you can pass returnJson as the second parameter:
ToonConverter.toJson(toonString, returnJson?)
ToonConverter.toJsonAsync(toonString, returnJson?)
The library now includes specialized, unified converters for each major format. These are perfect when you need to convert between non-TOON formats (like XML to JSON or CSV to YAML) while still having access to TOON and encryption features.
JsonConverter: Specialized in JSON input/outputYamlConverter: Specialized in YAML input/outputXmlConverter: Specialized in XML input/outputCsvConverter: Specialized in CSV input/outputimport { XmlConverter, YamlConverter } from 'toon-formatter';
// Convert XML directly to YAML
const xmlData = '<user><name>Alice</name></user>';
const yamlData = XmlConverter.toYaml(xmlData);
// Convert YAML directly to CSV
const csvData = YamlConverter.toCsv("name: Alice\nrole: admin");
Specialized for JSON-centric workflows. It can convert JSON to any format and any format back to JSON.
import { JsonConverter, Encryptor } from 'toon-formatter';
const converter = new JsonConverter(new Encryptor(key, 'aes-256-gcm'));
// JSON -> TOON (Encrypted)
const encryptedToon = converter.toToon(data, { conversionMode: 'export' });
// XML -> JSON (Decrypted)
const jsonData = converter.fromXml(encryptedXml, { conversionMode: 'ingestion' });
// Convert TOON to JSON object
const obj = JsonConverter.fromToon(toonString);
// Convert TOON to JSON string
const json = JsonConverter.fromToon(toonString, { returnJson: true });
// Convert JSON to XML
const xml = JsonConverter.toXml({ name: "Alice" });
Available Methods:
fromToon(toonString, options?) / fromToonAsync(toonString, options?)toToon(data, options?) / toToonAsync(data, options?)fromYaml(yamlString, options?) / fromYamlAsync(yamlString, options?)toYaml(data, options?) / toYamlAsync(data, options?)fromXml(xmlString, options?) / fromXmlAsync(xmlString, options?)toXml(data, options?) / toXmlAsync(data, options?)fromCsv(csvString, options?) / fromCsvAsync(csvString, options?)toCsv(data, options?) / toCsvAsync(data, options?)Specialized for YAML workflows.
import { YamlConverter } from 'toon-formatter';
// YAML to TOON
const toon = YamlConverter.fromToon(testToon);
// YAML to JSON
const json = YamlConverter.toJson(yamlString, { returnJson: true });
Available Methods:
fromToon(toonString, options?) / fromToonAsync(toonString, options?)toToon(yamlString, options?) / toToonAsync(yamlString, options?)fromJson(jsonData, options?) / fromJsonAsync(jsonData, options?)toJson(yamlString, options?) / toJsonAsync(yamlString, options?)fromXml(xmlString, options?) / fromXmlAsync(xmlString, options?)toXml(yamlString, options?) / toXmlAsync(yamlString, options?)fromCsv(csvString, options?) / fromCsvAsync(csvString, options?)toCsv(yamlString, options?) / toCsvAsync(yamlString, options?)Specialized for XML workflows. Supports mixed-text extraction automatically.
import { XmlConverter } from 'toon-formatter';
// XML to TOON
const toon = XmlConverter.fromToon(xmlString);
// JSON to XML
const xml = XmlConverter.fromJson(jsonData);
Available Methods:
fromToon(toonString, options?) / fromToonAsync(toonString, options?)toToon(xmlString, options?) / toToonAsync(xmlString, options?)fromJson(jsonData, options?) / fromJsonAsync(jsonData, options?)toJson(xmlString, options?) / toJsonAsync(xmlString, options?)fromYaml(yamlString, options?) / fromYamlAsync(yamlString, options?)toYaml(xmlString, options?) / toYamlAsync(xmlString, options?)fromCsv(csvString, options?) / fromCsvAsync(csvString, options?)toCsv(xmlString, options?) / toCsvAsync(xmlString, options?)Specialized for CSV workflows.
import { CsvConverter } from 'toon-formatter';
// CSV to TOON
const toon = CsvConverter.fromToon(csvString);
// JSON to CSV
const csv = CsvConverter.fromJson(jsonData);
Available Methods:
fromToon(toonString, options?) / fromToonAsync(toonString, options?)toToon(csvString, options?) / toToonAsync(csvString, options?)fromJson(jsonData, options?) / fromJsonAsync(jsonData, options?)toJson(csvString, options?) / toJsonAsync(csvString, options?)fromYaml(yamlString, options?) / fromYamlAsync(yamlString, options?)toYaml(csvString, options?) / toYamlAsync(csvString, options?)fromXml(xmlString, options?) / fromXmlAsync(xmlString, options?)toXml(csvString, options?) / toXmlAsync(csvString, options?)name: "Alice"
age: 30
active: true
score: null
user:
name: "Alice"
age: 30
numbers[3]: 1, 2, 3
names[2]: "Alice", "Bob"
items[2]:
- "First"
- "Second"
users[3]{id,name,active}:
1,"Alice",true
2,"Bob",false
3,"Charlie",true
company:
name: "TechCorp"
employees[2]:
-
name: "Alice"
role: "Engineer"
-
name: "Bob"
role: "Designer"
import { jsonToToonSync } from 'toon-formatter';
// Before: Sending JSON to LLM
const jsonPrompt = JSON.stringify(largeDataset);
// 1000+ tokens
// After: Sending TOON to LLM
const toonPrompt = jsonToToonSync(largeDataset);
// 600 tokens (40% reduction!)
const response = await openai.chat.completions.create({
messages: [{ role: "user", content: toonPrompt }]
});
import { jsonToToon, toonToJson } from 'toon-formatter';
// Convert to TOON before sending
const toonPrompt = await jsonToToon(largeDataset);
const response = await openai.chat.completions.create({
messages: [{ role: "user", content: toonPrompt }]
});
// Parse TOON response back to JSON
const result = await toonToJson(response.choices[0].message.content);
import { csvToToonSync, toonToJsonSync } from 'toon-formatter';
// Read CSV, convert to TOON, process, convert back
const csvData = fs.readFileSync('data.csv', 'utf-8');
const toonData = csvToToonSync(csvData);
// Send to LLM for processing...
const processedToon = await processWithLLM(toonData);
// Convert back to JSON for your app
const jsonResult = toonToJsonSync(processedToon);
import { jsonToToonSync, xmlToToon } from 'toon-formatter';
// Extract and convert JSON from API documentation
const apiDocs = `
The user endpoint returns:
{"id": 123, "name": "Alice", "email": "alice@example.com"}
The product endpoint returns:
{"id": 456, "title": "Widget", "price": 29.99}
`;
const convertedDocs = jsonToToonSync(apiDocs);
// Both JSON objects are converted to TOON while preserving the text
// Extract and convert XML from mixed content
const xmlContent = `
Server response:
<response><status>success</status><data>processed</data></response>
`;
const result = await xmlToToon(xmlContent);
// XML is converted to TOON format
import { yamlToToonSync, toonToYamlSync } from 'toon-formatter';
// Convert YAML config to TOON for LLM analysis
const yamlConfig = fs.readFileSync('config.yaml', 'utf-8');
const toonConfig = yamlToToonSync(yamlConfig);
// LLM can analyze and suggest improvements...
const improvedToon = await analyzeWithLLM(toonConfig);
// Convert back to YAML
const improvedYaml = toonToYamlSync(improvedToon);
fs.writeFileSync('config.yaml', improvedYaml);
The library includes a comprehensive test suite with 150+ unit and integration tests.
# Run all tests (Unit, Integration, and CLI)
npm test
MIT License - see LICENSE file for details
Contributions are welcome! Please feel free to submit a Pull Request.
| Format | Tokens | Reduction |
|---|---|---|
| JSON | 1000 | 0% |
| TOON | 600 | 40% |
Based on average structured data with arrays of objects
xmldom: npm install xmldomMade with ❤️ by Ankit Pal
FAQs
A powerful library to convert between TOON, JSON, YAML, XML, and CSV with end-to-end encryption. Includes Unified Converters for direct format-to-format translation and LLM token optimization.
The npm package toon-formatter receives a total of 15 weekly downloads. As such, toon-formatter popularity was classified as not popular.
We found that toon-formatter demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
OpenAI rotated macOS signing certificates after a malicious Axios package reached its CI pipeline in a broader software supply chain attack.
Security News
Open source is under attack because of how much value it creates. It has been the foundation of every major software innovation for the last three decades. This is not the time to walk away from it.
Security News
Socket CEO Feross Aboukhadijeh breaks down how North Korea hijacked Axios and what it means for the future of software supply chain security.