nodejs-rag-module
Advanced tools
A Node.js TypeScript module for RAG functionality using nomic-embed-text-v1.5 and sqlite-vec
A powerful Node.js TypeScript module for Retrieval-Augmented Generation (RAG) that provides text embedding, vector storage, and semantic search capabilities. Built with nomic-embed-text-v1.5 model and sqlite-vec for efficient vector operations.
npm install nodejs-rag-module
git clone <repository-url>
cd embedding-vec2-o
npm install
npm run build
The module automatically installs these core dependencies:
{
"sqlite3": "^5.1.6",
"sqlite-vec": "^0.1.7-alpha.2",
"node-llama-cpp": "^3.10.0",
"mysql2": "^3.14.2",
"pg": "^8.11.3"
}
import { RAGModule } from 'nodejs-rag-module';
const rag = new RAGModule({
modelPath: './nomic-embed-text-v1.5.Q5_K_M.gguf', // Path to embedding model
databasePath: './vectors.db', // SQLite database path
logLevel: 'info' // Logging level
});
| Option | Type | Default | Description |
|---|---|---|---|
modelPath | string | './nomic-embed-text-v1.5.Q5_K_M.gguf' | Path to the embedding model file |
databasePath | string | './rag.db' | Path to SQLite database file |
logLevel | string | 'info' | Logging level: 'debug', 'info', 'warn', 'error' |
import { HybridRAGModule } from 'nodejs-rag-module/services/hybridRAG';
const hybridRAG = new HybridRAGModule({
modelPath: './nomic-embed-text-v1.5.Q5_K_M.gguf',
databasePath: './vectors.db',
mysql: {
host: 'localhost',
port: 3306,
user: 'your_username',
password: 'your_password',
database: 'your_database',
connectionLimit: 10
}
});
import { PostgresHybridRAGModule } from 'nodejs-rag-module/services/postgresHybridRAG';
const postgresRAG = new PostgresHybridRAGModule({
modelPath: './nomic-embed-text-v1.5.Q5_K_M.gguf',
databasePath: './vectors.db',
postgres: {
host: 'localhost',
port: 5432,
user: 'your_username',
password: 'your_password',
database: 'your_database',
max: 10
}
});
import { RAGModule } from 'nodejs-rag-module';
const rag = new RAGModule({
modelPath: './nomic-embed-text-v1.5.Q5_K_M.gguf',
databasePath: './my-vectors.db'
});
// Initialize (required before use)
await rag.initialize();
// Save text with embeddings
await rag.save({
id: 1,
text: 'Machine learning is a subset of artificial intelligence',
tablename: 'documents'
});
await rag.save({
id: 2,
text: 'Deep learning uses neural networks with multiple layers',
tablename: 'documents'
});
// Find semantically similar documents
const results = await rag.search({
text: 'artificial intelligence and neural networks',
tablename: 'documents',
qty: 5
});
console.log('Similar document IDs:', results); // [1, 2]
// Generate embedding vector for any text
const vector = await rag.embed({
text: 'This is a sample text for embedding'
});
console.log('Vector dimensions:', vector.length); // 768
console.log('First 5 values:', vector.slice(0, 5));
// Close connections and free resources
await rag.close();
new RAGModule(config?: RAGModuleConfig)
initialize(): Promise<void>Initializes the embedding model and database connection. Must be called before using other methods.
embed(request: EmbedRequest): Promise<number[]>Generates vector embedding for text without storing it.
const vector = await rag.embed({ text: 'Hello world' });
save(request: SaveRequest): Promise<boolean>Embeds text and saves it with vector to database.
const success = await rag.save({
id: 1,
text: 'Document content',
tablename: 'my_documents'
});
search(request: SearchRequest): Promise<number[]>Searches for semantically similar texts and returns matching IDs.
const results = await rag.search({
text: 'search query',
tablename: 'my_documents',
qty: 10
});
close(): Promise<void>Closes all connections and frees resources.
interface RAGModuleConfig {
modelPath?: string;
databasePath?: string;
logLevel?: 'debug' | 'info' | 'warn' | 'error';
}
interface EmbedRequest {
text: string;
}
interface SaveRequest {
id: number;
text: string;
tablename: string;
}
interface SearchRequest {
text: string;
tablename: string;
qty: number;
}
| Database | Vector Storage | Document Storage | Metadata | Full-text Search | Connection Pooling |
|---|---|---|---|---|---|
| SQLite | ✅ sqlite-vec | ❌ | ❌ | ❌ | ❌ |
| MySQL | ❌ | ✅ | ✅ JSON | ❌ | ✅ |
| PostgreSQL | ❌ | ✅ | ✅ JSONB | ✅ Built-in | ✅ |
const rag = new RAGModule({
databasePath: './my-vectors.db'
});
const hybridRAG = new HybridRAGModule({
mysql: {
host: 'localhost',
user: 'username',
password: 'password',
database: 'my_database'
}
});
const postgresRAG = new PostgresHybridRAGModule({
postgres: {
host: 'localhost',
user: 'username',
password: 'password',
database: 'my_database'
}
});
The module uses nomic-embed-text-v1.5 by default, which provides:
# Download from Hugging Face
wget https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/resolve/main/nomic-embed-text-v1.5.Q5_K_M.gguf
# Using curl
curl -L -o nomic-embed-text-v1.5.Q5_K_M.gguf \
"https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/resolve/main/nomic-embed-text-v1.5.Q5_K_M.gguf"
The module works with any GGUF-format embedding model:
const rag = new RAGModule({
modelPath: './your-custom-model.gguf'
});
The same embedding model used to embed text into the vector database MUST be used for searching. Using different models will result in incompatible vector spaces and poor search results.
// ✅ Correct: Same model for embedding and searching
const rag = new RAGModule({ modelPath: './model-v1.5.gguf' });
await rag.save({ id: 1, text: 'document', tablename: 'docs' });
const results = await rag.search({ text: 'query', tablename: 'docs', qty: 5 });
// ❌ Incorrect: Different models will not work properly
const rag1 = new RAGModule({ modelPath: './model-v1.5.gguf' });
const rag2 = new RAGModule({ modelPath: './different-model.gguf' });
await rag1.save({ id: 1, text: 'document', tablename: 'docs' });
await rag2.search({ text: 'query', tablename: 'docs', qty: 5 }); // Poor results!
Build intelligent document search for:
// Index research papers
await rag.save({
id: 1,
text: 'Attention mechanisms in transformer architectures enable...',
tablename: 'research_papers'
});
// Find related papers
const similar = await rag.search({
text: 'transformer attention mechanisms',
tablename: 'research_papers',
qty: 10
});
import { RAGModule } from 'nodejs-rag-module';
async function ragWorkflow() {
const rag = new RAGModule({
modelPath: './nomic-embed-text-v1.5.Q5_K_M.gguf',
databasePath: './knowledge-base.db',
logLevel: 'info'
});
try {
// Initialize
await rag.initialize();
// Add knowledge base articles
const articles = [
{ id: 1, text: 'Machine learning algorithms learn patterns from data' },
{ id: 2, text: 'Neural networks are inspired by biological neurons' },
{ id: 3, text: 'Deep learning uses multiple layers for complex patterns' },
{ id: 4, text: 'Natural language processing helps computers understand text' }
];
// Save all articles
for (const article of articles) {
await rag.save({
id: article.id,
text: article.text,
tablename: 'knowledge_base'
});
}
// Search for relevant articles
const results = await rag.search({
text: 'How do neural networks work?',
tablename: 'knowledge_base',
qty: 3
});
console.log('Most relevant articles:', results);
} finally {
await rag.close();
}
}
import { HybridRAGModule } from 'nodejs-rag-module/services/hybridRAG';
async function hybridExample() {
const hybridRAG = new HybridRAGModule({
modelPath: './nomic-embed-text-v1.5.Q5_K_M.gguf',
databasePath: './vectors.db',
mysql: {
host: 'localhost',
user: 'root',
password: 'password',
database: 'documents'
}
});
await hybridRAG.initialize();
// Save document with rich metadata
const docId = await hybridRAG.saveDocument({
title: 'AI Research Paper',
content: 'Artificial intelligence research focuses on creating intelligent machines...',
metadata: {
author: 'Dr. Smith',
category: 'AI',
tags: ['machine learning', 'research'],
published: '2024-01-15'
},
tablename: 'research_papers'
});
// Search with full document data
const results = await hybridRAG.searchDocuments({
text: 'artificial intelligence research',
tablename: 'research_papers',
qty: 5,
includeContent: true,
includeMetadata: true
});
console.log('Search results with metadata:', results);
await hybridRAG.close();
}
async function batchProcessing() {
const rag = new RAGModule();
await rag.initialize();
// Process large dataset
const documents = await loadLargeDataset(); // Your data loading function
for (let i = 0; i < documents.length; i += 100) {
const batch = documents.slice(i, i + 100);
// Process in batches to manage memory
for (const doc of batch) {
await rag.save({
id: doc.id,
text: doc.content,
tablename: 'large_dataset'
});
}
console.log(`Processed ${i + batch.length} documents`);
}
await rag.close();
}
We welcome contributions! Please see our Contributing Guide for details.
git clone <repository-url>
cd embedding-vec2-o
npm install
npm run build
npm test
# Run all tests
npm test
# Run with coverage
npm run test:coverage
# Run specific test file
npm test -- database.test.ts
This project is licensed under the MIT License - see the LICENSE file for details.
Built with ❤️ for the AI community
FAQs
A Node.js TypeScript module for RAG functionality using nomic-embed-text-v1.5 and sqlite-vec
The npm package nodejs-rag-module receives a total of 0 weekly downloads. As such, nodejs-rag-module popularity was classified as not popular.
We found that nodejs-rag-module demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Multiple high-impact npm maintainers confirm they have been targeted in the same social engineering campaign that compromised Axios.
Security News
Axios compromise traced to social engineering, showing how attacks on maintainers can bypass controls and expose the broader software supply chain.
Security News
Node.js has paused its bug bounty program after funding ended, removing payouts for vulnerability reports but keeping its security process unchanged.