@unravel-tech/rlm-cli

loop-infer

A Clojure implementation of Recursive Language Models (RLMs) - a task-agnostic inference paradigm that enables LLMs to handle arbitrarily long contexts by storing data in a sandboxed REPL environment.

Based on the paper "Recursive Language Models" by Zhang, Kraska, and Khattab.

How It Works

Instead of passing large contexts directly to the model, RLMs:

• Store context in a sandboxed Clojure REPL
• Let the LLM write code to examine, decompose, and query the context
• Execute code safely and return results to the LLM
• Repeat until the LLM produces a final answer

The LLM can also spawn recursive LLM calls for analysis of context chunks.

Installation

Via npm (Recommended)

# Run directly without installation
npx @unravel-tech/rlm-cli -c document.txt -q "Summarize this"

# Or install globally
npm install -g @unravel-tech/rlm-cli
rlm-cli -c document.txt -q "Summarize this"

From Source

Prerequisites:

• Java 21+
• Clojure CLI

git clone https://github.com/unravel-team/loop-infer.git
cd loop-infer

Set your API key:

export OPENAI_API_KEY="sk-..."
# or
export ANTHROPIC_API_KEY="sk-ant-..."

CLI Usage

# Query a file
rlm -c document.txt -q "Summarize the key points"

# Query from stdin
cat large-file.txt | rlm -c - -q "Find all dates mentioned"

# Inline context
rlm -C "The capital of France is Paris." -q "What is the capital of France?"

# Query a directory of data files
rlm -d ./data -e parquet,csv -q "What patterns do you see?"

# Use a different model
rlm -m anthropic/claude-sonnet-4-20250514 -c doc.txt -q "Explain the main argument"

# Generate an HTML execution report
rlm -c doc.txt -q "Summarize" -H report.html

# Verbose output with JSON format
rlm -c doc.txt -q "Summarize" -v -o json

Server Mode

RLM can run as an HTTP server with real-time streaming support:

# Start server on default port 8080
rlm --server

# Start on custom port
rlm --server --port 3000

HTTP API Endpoints

Endpoint	Method	Description
/health	GET	Health check, returns {"status": "ok", "active-tasks": N}
/api/query	POST	Synchronous query, returns JSON result
/api/stream	POST	Streaming query with SSE progress events
/api/task/:id	GET	Get task status
/api/task/:id/report	GET	Get HTML execution report
/api/task/:id/live	GET	Live-updating HTML report page

Query Example

curl -X POST http://localhost:8080/api/query \
  -H "Content-Type: application/json" \
  -d '{
    "context": "The quick brown fox jumps over the lazy dog.",
    "query": "What animals are mentioned?",
    "model": "openai/gpt-4o",
    "trace": true
  }'

Response:

{
  "answer": "The animals mentioned are a fox and a dog.",
  "costs": {
    "total-cost": 0.00123,
    "root-llm-calls": 2,
    "recursive-llm-calls": 0
  },
  "trace": { ... }
}

Streaming Example

curl -X POST http://localhost:8080/api/stream \
  -H "Content-Type: application/json" \
  -d '{"context": "Hello world", "query": "What is this?"}'

Returns Server-Sent Events (SSE) with real-time progress:

• connected - Task created, returns task ID
• start - Query execution started
• iteration-start - New iteration beginning
• llm-call - Root LLM request/response
• repl-execution - Code execution result
• recursive-llm - Recursive LLM call
• result - Final answer with costs
• done - Execution complete

Live Report

Open http://localhost:8080/api/task/{task-id}/live in a browser to see a real-time updating execution report.

Library Usage

(require '[loop-infer.core :as rlm])

;; Create an RLM instance
(def r (rlm/create-rlm {:model "openai/gpt-4o"
                        :trace true
                        :verbose true}))

;; Query with context
(rlm/query r {:context "Paris is the capital of France."
              :query "What is the capital of France?"})
;; => "Paris"

;; Check costs
(rlm/cost-summary r)
;; => {:total-cost 0.00123 :root-llm-cost 0.001 :recursive-llm-cost 0.00023 ...}

;; Generate HTML report (if trace was enabled)
(rlm/write-html-report r "report.html")

;; Reset for reuse
(rlm/reset-rlm! r)

CLI Options

Flag	Default	Description
-m, --model	openai/gpt-4o	LLM model (provider/model format)
-r, --recursive-model	(same as --model)	Override model for recursive calls
-c, --context	-	Context file path (or - for stdin)
-C, --context-string	-	Inline context string
-d, --directory	-	Load files from directory as context
-p, --pattern	-	Glob pattern for directory files
-e, --extensions	-	Comma-separated file extensions
-n, --max-files	-	Maximum files to load
-q, --query	-	Query to answer (required in CLI mode)
-i, --max-iterations	20	Max RLM iterations
-v, --verbose	false	Enable verbose output
-o, --output	text	Output format: text, json, edn
-H, --html-report	-	Generate HTML report to file
-S, --server	false	Run as HTTP server
-P, --port	8080	Server port
--host	0.0.0.0	Server host to bind
-k, --api-key	-	API key (or use env vars)

Supported Models

Any model supported by litellm-clj using provider/model-name format:

• OpenAI: openai/gpt-4o, openai/gpt-4o-mini
• Anthropic: anthropic/claude-sonnet-4-20250514, anthropic/claude-3-5-haiku-20241022

Supported File Types

• Text: txt, md, rst, html, xml, yaml, json, py, js, clj, etc.
• Data: parquet, csv, tsv, json, jsonl, arrow

Building from Source

# Build uberjar
./scripts/build.sh
# or
clojure -X:uberjar

# Run the jar
java -jar loop-infer.jar --help

Publishing to npm

# Build uberjar first
./scripts/build.sh

# Install jdeploy and publish
npm install -g jdeploy
jdeploy publish

Running Tests

clojure -M:test

Architecture

src/loop_infer/
├── core.clj        # Public API and CLI
├── rlm.clj         # Core RLM iteration algorithm
├── repl.clj        # Sandboxed REPL execution
├── prompts.clj     # System prompts for LLMs
├── server.clj      # HTTP server with SSE streaming
├── trace.clj       # Execution tracing and HTML reports
└── file-loader.clj # File and directory loading

See implementation.md for detailed architecture documentation.

License

MIT