rails-llm-integration

Production-grade LLM integration patterns for Ruby on Rails applications. USE THIS SKILL when the user needs to: integrate OpenAI, Claude, or any LLM API into a Rails app; use ruby_llm gem or langchain-rb gem in Rails; build AI-powered features in Rails; manage prompts as templates; route between LLM models for cost optimization; run LLM calls as background jobs; set up Braintrust evals; use LiteLLM or Portkey as a proxy; track LLM costs; test AI features; audit LLM usage patterns; build RAG pipelines in Rails with langchain-rb; add classification, summarization, extraction, or generation features to Rails; manage LLM API keys and budgets; handle LLM errors and retries; or adopt conventions for LLM service objects similar to ActionMailer or ActiveJob patterns. Triggers on: LLM in Rails, OpenAI Rails integration, Claude API Ruby, Anthropic SDK Rails, AI features Rails, prompt engineering Ruby, model routing, LLM cost optimization, Braintrust Rails, eval pipeline Ruby, background LLM jobs, async AI calls, LLM service objects, prompt templates Rails, ruby-openai gem, anthropic-rb gem, ruby-llm gem, ruby_llm gem, langchain-rb gem, langchainrb, faraday LLM, LLM anti-patterns Rails, audit LLM calls, AI service objects, Rails AI conventions, RAG Rails, vector search Rails, pgvector Rails.

rubyonai 13 Updated 2mo ago

GitHub

Install

npx skillscat add rubyonai/rails-llm-integration

Install via the SkillsCat registry.

SKILL.md

Rails LLM Integration Skill

What This Skill Does

Teaches Claude to write LLM integrations the Rails Way -- treating LLM calls like
first-class Rails citizens alongside ActionMailer (email), ActiveJob (background work),
and ActionController (HTTP). Instead of scattered API calls, you get conventions.

When to Use

Adding any AI/LLM feature to a Rails application
Using ruby_llm gem, langchain-rb gem, ruby-openai, or anthropic-rb in Rails
Setting up prompt management, model routing, or cost tracking
Building eval pipelines with Braintrust
Building RAG pipelines with langchain-rb and pgvector
Auditing existing LLM usage for anti-patterns
Testing AI-powered features

Architecture Overview

app/
  services/
    llm/
      base_service.rb          # All LLM calls inherit from this
      concerns/                # Traceable, Retryable, CostTrackable
      product_description_service.rb
      ticket_triage_service.rb
      errors.rb                # Typed error hierarchy
      result.rb                # Result monad
  jobs/
    llm/
      base_job.rb              # Async LLM calls inherit from this
      generate_description_job.rb
  prompts/                     # Prompts as views (ERB templates)
    product_descriptions/
      generate.text.erb
      generate.system.erb
    ticket_triage/
      classify.text.erb
config/
  llm.yml                     # Model config like database.yml
  initializers/
    llm.rb                    # Boot-time config validation
lib/
  llm/
    client.rb                 # Client factory: LLM::Client.for(model)
    clients/
      openai_client.rb        # Wraps ruby-openai gem
      anthropic_client.rb     # Wraps anthropic-rb gem
      ruby_llm_client.rb      # Wraps ruby_llm gem (multi-provider)
      langchain_client.rb     # Wraps langchain-rb gem
      proxy_client.rb         # LiteLLM/Portkey via Faraday
      stub_client.rb          # Test double
    config.rb                 # Loads config/llm.yml
    router.rb                 # Task-based model routing
    cost_tracker.rb           # Redis-backed cost tracking
    prompt_renderer.rb        # ERB-based prompt rendering
db/
  migrate/
    create_llm_batches.rb     # Batch job tracking
    create_llm_dead_letters.rb # Permanent failure tracking
    create_llm_eval_cases.rb  # Eval dataset storage
    create_llm_experiment_logs.rb # Shadow experiment results

Installation

rails generate llm:install       # Sets up everything above
rails generate llm:service ProductDescription generation  # Scaffold a new service

Reference Files

File	Read When You Need To...
`references/client-setup.md`	Wire up ruby_llm, langchain-rb, ruby-openai, anthropic-rb, or proxy clients (start here)
`references/service-patterns.md`	Build LLM service objects with Result, tracing, retries
`references/job-patterns.md`	Make LLM calls async with ActiveJob + Sidekiq
`references/proxy-routing.md`	Route between models, optimize costs, set budgets
`references/eval-pipeline.md`	Set up Braintrust evals and quality scoring
`references/prompt-management.md`	Manage prompts as ERB templates (prompts-as-views)
`references/testing-guide.md`	Test LLM features with stubs, VCR, and CI strategy
`references/generators.md`	Rails generators for `llm:install` and `llm:service`
`templates/migrations/`	Database schemas for batches, dead letters, eval cases
`scripts/audit_llm_usage.rb`	Scan a Rails app for LLM anti-patterns

Core Principles

Rails conventions wrap LLM libraries -- ruby_llm and langchain-rb are the engine,
Rails patterns are the chassis. Use service objects, ActiveJob, ERB templates, and
YAML config. The gem handles the API; the convention handles everything else.
Async by default -- Every LLM call goes through ActiveJob unless the user is
actively waiting (streaming). Never block a web request on a 3-second API call.
Cost-aware routing -- Every call has an estimated cost. Route cheap tasks to
cheap models. Set daily budgets. Alert before you hit them.
Eval-first development -- Log every call with Braintrust. Build eval datasets
from production traces. Score quality automatically. Gate deployments on eval
regressions.
Prompts are views -- Prompts live in app/prompts/ as ERB templates, versioned
in git, rendered with locals, tested independently from API calls.
Errors are typed -- RateLimitError, TimeoutError, ContentFilterError,
BudgetExceededError. Each has its own retry and alerting strategy.

rails-llm-integration

Install

Rails LLM Integration Skill

What This Skill Does

When to Use

Architecture Overview

Installation

Reference Files

Core Principles

Categories

Install

Recommended Skills