Rails LLM Integration Skill

What This Skill Does

Teaches Claude to write LLM integrations the Rails Way -- treating LLM calls like first-class Rails citizens alongside ActionMailer (email), ActiveJob (background work), and ActionController (HTTP). Instead of scattered API calls, you get conventions.

When to Use

• Adding any AI/LLM feature to a Rails application
• Using ruby_llm gem, langchain-rb gem, ruby-openai, or anthropic-rb in Rails
• Setting up prompt management, model routing, or cost tracking
• Building eval pipelines with Braintrust
• Building RAG pipelines with langchain-rb and pgvector
• Auditing existing LLM usage for anti-patterns
• Testing AI-powered features

Architecture Overview

app/
  services/
    llm/
      base_service.rb          # All LLM calls inherit from this
      concerns/                # Traceable, Retryable, CostTrackable
      product_description_service.rb
      ticket_triage_service.rb
      errors.rb                # Typed error hierarchy
      result.rb                # Result monad
  jobs/
    llm/
      base_job.rb              # Async LLM calls inherit from this
      generate_description_job.rb
  prompts/                     # Prompts as views (ERB templates)
    product_descriptions/
      generate.text.erb
      generate.system.erb
    ticket_triage/
      classify.text.erb
config/
  llm.yml                     # Model config like database.yml
  initializers/
    llm.rb                    # Boot-time config validation
lib/
  llm/
    client.rb                 # Client factory: LLM::Client.for(model)
    clients/
      openai_client.rb        # Wraps ruby-openai gem
      anthropic_client.rb     # Wraps anthropic-rb gem
      ruby_llm_client.rb      # Wraps ruby_llm gem (multi-provider)
      langchain_client.rb     # Wraps langchain-rb gem
      proxy_client.rb         # LiteLLM/Portkey via Faraday
      stub_client.rb          # Test double
    config.rb                 # Loads config/llm.yml
    router.rb                 # Task-based model routing
    cost_tracker.rb           # Redis-backed cost tracking
    prompt_renderer.rb        # ERB-based prompt rendering
db/
  migrate/
    create_llm_batches.rb     # Batch job tracking
    create_llm_dead_letters.rb # Permanent failure tracking
    create_llm_eval_cases.rb  # Eval dataset storage
    create_llm_experiment_logs.rb # Shadow experiment results

Installation

rails generate llm:install       # Sets up everything above
rails generate llm:service ProductDescription generation  # Scaffold a new service

Reference Files

Core Principles

1. Rails conventions wrap LLM libraries -- ruby_llm and langchain-rb are the engine, Rails patterns are the chassis. Use service objects, ActiveJob, ERB templates, and YAML config. The gem handles the API; the convention handles everything else.

2. Async by default -- Every LLM call goes through ActiveJob unless the user is actively waiting (streaming). Never block a web request on a 3-second API call.

3. Cost-aware routing -- Every call has an estimated cost. Route cheap tasks to cheap models. Set daily budgets. Alert before you hit them.

4. Eval-first development -- Log every call with Braintrust. Build eval datasets from production traces. Score quality automatically. Gate deployments on eval regressions.

5. Prompts are views -- Prompts live in app/prompts/ as ERB templates, versioned in git, rendered with locals, tested independently from API calls.

6. Errors are typed -- RateLimitError, TimeoutError, ContentFilterError, BudgetExceededError. Each has its own retry and alerting strategy.