@ruvector/ruvllm

@ruvector/ruvllm

Build AI that learns and improves from every interaction.

RuvLLM is a self-learning language model toolkit that gets smarter over time. Unlike traditional LLMs that remain static after training, RuvLLM continuously adapts to your use case while remembering what it learned before.

What Makes RuvLLM Different?

Traditional LLMs forget old knowledge when learning new things (called "catastrophic forgetting"). RuvLLM solves this with three key innovations:

• It Learns Without Forgetting - Uses tiny parameter updates (LoRA) and memory protection (EWC++) to learn new patterns while preserving existing knowledge

• It Remembers Context - Built-in vector memory stores and retrieves relevant information instantly using similarity search

• It Routes Intelligently - Automatically selects the right model size and parameters based on query complexity, saving resources on simple tasks

Key Features

Feature	What It Does	Why It Matters
Adaptive Learning	Learns from user feedback in real-time	Improves accuracy over time without retraining
Memory System	Stores context with instant similarity search	Finds relevant information in microseconds
Smart Routing	Picks optimal model/settings per query	Reduces costs, improves response quality
SIMD Acceleration	Uses CPU vector instructions (AVX2/NEON)	10-50x faster vector operations
Federated Learning	Train across devices without sharing data	Privacy-preserving distributed learning
LoRA Adapters	Parameter-efficient fine-tuning with low-rank matrices	Fast adaptation with minimal memory
EWC++ Protection	Elastic Weight Consolidation prevents forgetting	Learn new tasks without losing old knowledge
SafeTensors Export	HuggingFace-compatible model serialization	Share models with the ML ecosystem
Training Pipeline	Full training infrastructure with schedulers	Production-ready model training
Session Management	Stateful conversations with streaming	Build chat applications easily

Installation

npm install @ruvector/ruvllm

Or run directly:

npx @ruvector/ruvllm info

Quick Start Tutorial

1. Basic Query

import { RuvLLM } from '@ruvector/ruvllm';

const llm = new RuvLLM();

// Ask a question - routing happens automatically
const response = llm.query('Explain neural networks simply');
console.log(response.text);
// Output: "Neural networks are computing systems inspired by..."

console.log(`Used model: ${response.model}`);
console.log(`Confidence: ${(response.confidence * 100).toFixed(1)}%`);

2. Teaching the System

// Query and get a response
const response = llm.query('What is the capital of France?');

// Provide feedback - the system learns from this
llm.feedback({
  requestId: response.requestId,
  rating: 5,  // 1-5 scale
  correction: 'Paris is the capital and largest city of France'
});

// Future similar queries will be more accurate

3. Using Memory

// Store important context
llm.addMemory('Company policy: All returns accepted within 30 days', {
  category: 'policy',
  department: 'customer-service'
});

llm.addMemory('Product X launched in March 2024 with features A, B, C', {
  category: 'product',
  name: 'Product X'
});

// Search memory for relevant context
const results = llm.searchMemory('return policy', 5);
console.log(results[0].content);
// Output: "Company policy: All returns accepted within 30 days"
console.log(`Relevance: ${(results[0].score * 100).toFixed(1)}%`);

4. Computing Similarity

import { SimdOps } from '@ruvector/ruvllm';

const simd = new SimdOps();

// Compare two texts
const score = llm.similarity(
  'How do I reset my password?',
  'I forgot my login credentials'
);
console.log(`Similarity: ${(score * 100).toFixed(1)}%`);
// Output: "Similarity: 78.3%"

// Fast vector operations
const embedding1 = llm.embed('machine learning');
const embedding2 = llm.embed('deep learning');
const similarity = simd.cosineSimilarity(embedding1, embedding2);

5. Batch Processing

// Process multiple queries efficiently
const batch = llm.batchQuery({
  queries: [
    'What is AI?',
    'Explain machine learning',
    'How do neural networks work?'
  ],
  config: { temperature: 0.7 }
});

batch.responses.forEach((r, i) => {
  console.log(`Query ${i + 1}: ${r.text.slice(0, 50)}...`);
});
console.log(`Total time: ${batch.totalLatencyMs}ms`);

CLI Commands

# Get system information
ruvllm info

# Query the model
ruvllm query "What is quantum computing?"

# Generate text with custom settings
ruvllm generate "Write a product description for:" --temperature 0.8 --max-tokens 200

# Memory operations
ruvllm memory add "Important fact to remember"
ruvllm memory search "fact" --k 10

# Compare texts
ruvllm similarity "hello world" "hi there"

# Get embeddings
ruvllm embed "your text here"

# Run performance benchmark
ruvllm benchmark --dims 768 --iterations 5000

# View statistics
ruvllm stats --json

Benchmarks

Benchmarked in Docker (node:20-alpine, x64) - December 2024

Core Operations

Operation	Time	Throughput
Query (short)	1.49μs	670K ops/s
Query (long)	874ns	1.14M ops/s
Generate	88ns	11.4M ops/s
Route	92ns	10.9M ops/s
Embed (256d)	10.6μs	94K ops/s
Embed (768d)	7.1μs	140K ops/s

SIMD Vector Operations

Operation	128d	256d	512d	768d
Dot Product	214ns / 4.67M ops/s	318ns / 3.15M ops/s	609ns / 1.64M ops/s	908ns / 1.10M ops/s
Cosine Similarity	233ns / 4.30M ops/s	335ns / 2.99M ops/s	652ns / 1.53M ops/s	972ns / 1.03M ops/s
L2 Distance	195ns / 5.14M ops/s	315ns / 3.18M ops/s	612ns / 1.63M ops/s	929ns / 1.08M ops/s

LoRA Adapter Performance

Operation	64d	128d	256d
Forward (r=4)	6.09μs / 164K ops/s	2.74μs / 365K ops/s	4.83μs / 207K ops/s
Forward (r=8)	2.17μs / 462K ops/s	4.30μs / 233K ops/s	8.99μs / 111K ops/s
Forward (r=16)	4.85μs / 206K ops/s	9.05μs / 111K ops/s	18.3μs / 55K ops/s
Backward (r=8)	-	110μs / 9.1K ops/s	-
Batch (100)	-	467μs / 2.1K ops/s	-

Memory Operations

Operation	Time	Throughput
Add Memory	5.3μs	189K ops/s
Search (k=5)	45.6μs	21.9K ops/s
Search (k=10)	28.3μs	35.3K ops/s
Search (k=20)	33.1μs	30.2K ops/s

SONA Learning System

Operation	Time	Throughput
Pattern Store	14.4μs	69.5K ops/s
Pattern Find Similar	224μs	4.5K ops/s
EWC Register Task	6.5μs	154K ops/s
EWC Compute Penalty	501μs	2.0K ops/s
Trajectory Build	1.24μs	807K ops/s

Federated Learning

Operation	Time	Throughput
Agent Create	7.8μs	128K ops/s
Process Task	7.9μs	126K ops/s
Apply LoRA	12.6μs	79.6K ops/s
Export State	48.9μs	20.4K ops/s
Aggregate	5.26ms	190 ops/s

Session & Streaming

Operation	Time	Throughput
Session Create	1.45μs	690K ops/s
Session Chat	3.28μs	305K ops/s
Session Export	3.91ms	255 ops/s
Session Import	1.60ms	625 ops/s

Training Pipeline

Operation	Time
Pipeline Create	70.6μs
Add Data (100 samples)	70.6μs
Train (32 samples, 3 epochs)	1.33s

Export/Import

Operation	Time	Throughput
SafeTensors Write	67.3μs	14.9K ops/s
SafeTensors Read	102μs	9.8K ops/s
LoRA to JSON	87.9μs	11.4K ops/s
LoRA from JSON	86.0μs	11.6K ops/s

Performance Highlights

• Fastest: Generate at 11.4M ops/s, Route at 10.9M ops/s
• Vector Ops: Up to 5.14M ops/s for L2 distance (128d)
• LoRA Forward: Up to 462K ops/s (64d, rank-8)
• Memory Search: 35K ops/s (k=10)
• Session Create: 690K ops/s

Configuration

const llm = new RuvLLM({
  // Embedding settings
  embeddingDim: 768,        // Vector dimensions (384, 768, 1024)

  // Memory settings
  hnswM: 16,                // Graph connectivity (higher = better recall, more memory)
  hnswEfConstruction: 100,  // Build quality (higher = better index, slower build)
  hnswEfSearch: 64,         // Search quality (higher = better recall, slower search)

  // Learning settings
  learningEnabled: true,    // Enable adaptive learning
  qualityThreshold: 0.7,    // Min confidence to skip learning
  ewcLambda: 2000,          // Memory protection strength

  // Router settings
  routerHiddenDim: 128,     // Router network size
});

Platform Support

Native acceleration available on:

Platform	Architecture	SIMD Support
macOS	Apple Silicon (M1/M2/M3)	NEON
macOS	Intel x64	AVX2, SSE4.1
Linux	x64	AVX2, AVX-512, SSE4.1
Linux	ARM64	NEON
Windows	x64	AVX2, SSE4.1

Falls back to optimized JavaScript on unsupported platforms.

Real-World Use Cases

Customer Support Bot

// Store FAQ and policies
faqs.forEach(faq => llm.addMemory(faq.answer, { question: faq.question }));

// Answer questions with context
function answerQuestion(question: string) {
  const context = llm.searchMemory(question, 3);
  const prompt = `Context:\n${context.map(c => c.content).join('\n')}\n\nQuestion: ${question}`;
  return llm.query(prompt);
}

Document Search

// Index documents
documents.forEach(doc => {
  llm.addMemory(doc.content, {
    title: doc.title,
    path: doc.path
  });
});

// Semantic search
const results = llm.searchMemory('quarterly revenue growth', 10);

Personalized Recommendations

// Learn from user interactions
function recordInteraction(userId: string, itemId: string, rating: number) {
  const response = llm.query(`User ${userId} rated ${itemId}`);
  llm.feedback({ requestId: response.requestId, rating });
}

// Get recommendations
function recommend(userId: string) {
  return llm.searchMemory(`preferences for user ${userId}`, 10);
}

API Reference

RuvLLM Class

Method	Description
query(text, config?)	Query with automatic model routing
generate(prompt, config?)	Generate text with given prompt
route(text)	Get routing decision without executing
addMemory(content, metadata?)	Store content in vector memory
searchMemory(text, k?)	Find similar content (default k=10)
feedback(fb)	Submit feedback for learning
embed(text)	Get embedding vector for text
similarity(t1, t2)	Compute similarity between texts
stats()	Get engine statistics
forceLearn()	Trigger immediate learning cycle

SimdOps Class

Method	Description
dotProduct(a, b)	Vector dot product
cosineSimilarity(a, b)	Cosine similarity (0-1)
l2Distance(a, b)	Euclidean distance
normalize(v)	Normalize to unit length
softmax(v)	Softmax activation
relu(v)	ReLU activation
gelu(v)	GELU activation
layerNorm(v, eps?)	Layer normalization
matvec(m, v)	Matrix-vector multiply

Troubleshooting

Q: Native module not loading?

ruvllm info  # Check if native is loaded

If "Native: Fallback", install platform-specific package manually:

npm install @ruvector/ruvllm-darwin-arm64  # For Apple Silicon

Q: Memory usage too high? Reduce HNSW parameters:

const llm = new RuvLLM({ hnswM: 8, hnswEfConstruction: 50 });

Q: Learning not improving results? Check that feedback is being processed:

const stats = llm.stats();
console.log(`Patterns learned: ${stats.patternsLearned}`);

License

MIT OR Apache-2.0

Links

• GitHub Repository
• Documentation
• Issue Tracker