rlama

RLAMA - Local RAG System

RLAMA (Retrieval-Augmented Language Model Adapter) provides fully local, offline RAG for semantic search over your documents.

When to Use This Skill

• Building knowledge bases from local documents
• Searching personal notes, research papers, or code documentation
• Document-based Q&A without sending data to the cloud
• Indexing project documentation for quick semantic lookup
• Creating searchable archives of PDFs, markdown, or code files

Prerequisites

RLAMA requires Ollama running locally:

# Verify Ollama is running
ollama list

# If not running, start it
brew services start ollama  # macOS
# or: ollama serve

Quick Reference

Query a RAG (Default: Retrieve-Only)

Always use retrieve-only mode by default. Claude synthesizes far better answers than local 7B models. The raw chunks give Claude direct evidence to reason over and cite.

# DEFAULT: Retrieve top 10 chunks — Claude reads and synthesizes
python3 ~/.claude/skills/rlama/scripts/rlama_retrieve.py <rag-name> "your query"

# More chunks for broad queries
python3 ~/.claude/skills/rlama/scripts/rlama_retrieve.py <rag-name> "your query" -k 20

# JSON output for programmatic use
python3 ~/.claude/skills/rlama/scripts/rlama_retrieve.py <rag-name> "your query" --json

# Force rebuild embedding cache
python3 ~/.claude/skills/rlama/scripts/rlama_retrieve.py <rag-name> "your query" --rebuild-cache

# List RAGs with cache status
python3 ~/.claude/skills/rlama/scripts/rlama_retrieve.py --list

First run per collection builds an embedding cache (~60s for 4K chunks). Subsequent queries are <1s.

Local LLM Query (Fallback Only)

Use rlama run only when Claude is not in the loop (e.g., standalone CLI usage, cron jobs, scripts):

# Local model generates the answer (weaker than Claude synthesis)
rlama run <rag-name> --query "your question here"

# With more context chunks
rlama run <rag-name> --query "explain the authentication flow" --context-size 30

# Show source documents
rlama run <rag-name> --query "what are the API endpoints?" --show-context

Script wrapper for cleaner output:

python3 ~/.claude/skills/rlama/scripts/rlama_query.py <rag-name> "your query"
python3 ~/.claude/skills/rlama/scripts/rlama_query.py my-docs "what is the main idea?" --show-sources

External LLM Synthesis (optional—retrieve chunks AND synthesize via OpenRouter, TogetherAI, Ollama, or any OpenAI-compatible endpoint):

# Synthesize via OpenRouter (auto-detected from model with /)
python3 ~/.claude/skills/rlama/scripts/rlama_retrieve.py <rag-name> "your query" --synthesize --synth-model anthropic/claude-sonnet-4

# Synthesize via TogetherAI
python3 ~/.claude/skills/rlama/scripts/rlama_retrieve.py <rag-name> "your query" --synthesize --provider togetherai

# Synthesize via local Ollama (fully offline, uses research-grade system prompt)
python3 ~/.claude/skills/rlama/scripts/rlama_retrieve.py <rag-name> "your query" --synthesize --provider ollama

# Synthesize via custom endpoint
python3 ~/.claude/skills/rlama/scripts/rlama_retrieve.py <rag-name> "your query" --synthesize --endpoint https://my-api.com/v1/chat/completions

Environment variables for synthesis:

| Variable | Provider | |----------|----------| | OPENROUTER_API_KEY | OpenRouter (default, auto-detected first) | | TOGETHER_API_KEY | TogetherAI | | SYNTH_API_KEY | Custom endpoint (via --endpoint) | | (none needed) | Ollama (local, no auth) |

Provider auto-detection: model names with / → OpenRouter, otherwise → TogetherAI. Falls back to whichever API key is set.

Quality tiers:

| Tier | Method | Quality | Latency | Default? | |------|--------|---------|---------|----------| | Best | Retrieve-only → Claude synthesizes | Strongest synthesis | ~1s retrieve | YES | | Good | --synthesize --synth-model anthropic/claude-sonnet-4 | Strong, cited | ~3s | | | Decent | --synthesize --provider togetherai (Llama 70B) | Solid for factual | ~2s | | | Reasoning | --synthesize --reasoning (Qwen 3.5 9B) | Strong local, cited | ~8s | | | Local | --synthesize --provider ollama (Qwen 2.5 7B) | Basic, may hedge | ~5s | | | Baseline | rlama_query.py (RLAMA built-in) | Weakest, no prompt control | ~3s | |

Small local models (7B) use a tuned prompt optimized for Qwen (structured output, anti-hedge, domain-keyword aware). Cloud providers use a strict research-grade prompt with mandatory citations. Reasoning mode (--reasoning) uses qwen3.5:9b with the strict prompt and 4096 max tokens—best local option for complex cross-document synthesis.

First run builds an embedding cache (~30s for 3K chunks, ~10min for 25K chunks). Subsequent queries are <1s. Large RAGs use incremental checkpointing—if Ollama crashes mid-build, re-run to resume from the last checkpoint. Individual chunks are truncated to 5K chars to stay within the embedding model's context window.

Benchmarking:

# Retrieval quality only
python3 ~/.claude/skills/rlama/scripts/rlama_bench.py <rag-name> --retrieval-only

# Full synthesis benchmark (8 test cases)
python3 ~/.claude/skills/rlama/scripts/rlama_bench.py <rag-name> --provider ollama --verbose

# Single test case
python3 ~/.claude/skills/rlama/scripts/rlama_bench.py <rag-name> --provider ollama --case 0

# JSON output for analysis
python3 ~/.claude/skills/rlama/scripts/rlama_bench.py <rag-name> --provider ollama --json

Scores: retrieval precision, topic coverage, grounding, directness (anti-hedge), composite (0-100).

Create a RAG

Index documents from a folder into a new RAG system:

# Basic creation (uses llama3.2 by default)
rlama rag llama3.2 <rag-name> <folder-path>

# Examples
rlama rag llama3.2 my-notes ~/Notes
rlama rag llama3.2 project-docs ./docs
rlama rag llama3.2 research-papers ~/Papers

# With exclusions
rlama rag llama3.2 codebase ./src --exclude-dir=node_modules,dist,.git --exclude-ext=.log,.tmp

# Only specific file types
rlama rag llama3.2 markdown-docs ./docs --process-ext=.md,.txt

# Custom chunking strategy
rlama rag llama3.2 my-rag ./docs --chunking=semantic --chunk-size=1500 --chunk-overlap=300

Chunking strategies:

• hybrid (default) - Combines semantic and fixed chunking
• semantic - Respects document structure (paragraphs, sections)
• fixed - Fixed character count chunks
• hierarchical - Preserves document hierarchy

List RAG Systems

# List all RAGs
rlama list

# List documents in a specific RAG
rlama list-docs <rag-name>

# Inspect chunks (debugging)
rlama list-chunks <rag-name> --document=filename.pdf

Manage Documents

Add documents to existing RAG:

rlama add-docs <rag-name> <folder-or-file>

# Examples
rlama add-docs my-notes ~/Notes/new-notes
rlama add-docs research ./papers/new-paper.pdf

Remove a document:

rlama remove-doc <rag-name> <document-id>

# Document ID is typically the filename
rlama remove-doc my-notes old-note.md
rlama remove-doc research outdated-paper.pdf

# Force remove without confirmation
rlama remove-doc my-notes old-note.md --force

Delete a RAG

rlama delete <rag-name>

# Or manually remove the data directory
rm -rf ~/.rlama/<rag-name>

Advanced Features

Web Crawling

Create a RAG from website content:

# Crawl a website and create RAG
rlama crawl-rag llama3.2 docs-rag https://docs.example.com

# Add web content to existing RAG
rlama crawl-add-docs my-rag https://blog.example.com

Directory Watching

Automatically update RAG when files change:

# Enable watching
rlama watch <rag-name> <folder-path>

# Check for new files manually
rlama check-watched <rag-name>

# Disable watching
rlama watch-off <rag-name>

Website Watching

Monitor websites for content updates:

rlama web-watch <rag-name> https://docs.example.com
rlama check-web-watched <rag-name>
rlama web-watch-off <rag-name>

Reranking