The Ruby AI framework
chat = RubyLLM.chat
chat.ask "What's the best way to learn Ruby?"
chat.ask "What's in this image?", with: "ruby_conf.jpg"
chat.with_model("claude-sonnet-5").ask "Now say it as a haiku"
RubyLLM.paint("a sunset over mountains in watercolor").save("sunset.png")
Website · Getting Started · What's New in 2.2
Battle tested at - Fully private work AI
https://github.com/user-attachments/assets/519fcb20-ab8c-4af3-9185-0953b3ba7b81
[!NOTE]
Using RubyLLM? Share your story! Takes 5 minutes.
23 providers. One API.
Build with the models you want. Move between hosted and local providers without rewriting your application, or connect an OpenAI-compatible endpoint.
Browse models and pricing · Connect a provider
A complete AI framework for Ruby
More than a client for model APIs. RubyLLM gives your application the conventions, the runtime, and every piece in between, designed to work together. Everything below ships in RubyLLM 2.2.
Conversations
- Chat: Conversational AI with
RubyLLM.chat
- Streaming: Real-time responses with blocks
- Attachments: Ask about images, videos, audio, PDFs, and other documents with
with:
- Structured output: Define a Ruby schema and read the result with
response.parsed
- Extended thinking: Control, view, and persist model deliberation
- Citations: Normalized source citations from documents, search, and grounding
- Prompt caching: Turn on the provider's cache with
with_caching and cache_until_here
- Compaction: Let providers condense long conversations with
with_compaction
- Token counting: Count a request before you send it with
count_tokens
- Fallbacks and cancellation: Retry on backup models with
with_fallbacks, stop a run with cancel
Tools and agents
- Tools: Let AI call your Ruby methods
- Tool approval: Park a run until a human approves with
requires_approval
- Provider tools: Web search, code execution, and MCP connectors with
with_provider_tools
- MCP: Connect agents to your systems with server classes in
app/mcp, per-user inputs, tool filtering, and OAuth with RubyLLM::MCP
- Agents: Reusable assistants with
RubyLLM::Agent
- The agentic loop: Drive it yourself with
ask_later, step, and complete?
- Prompt templates: ERB prompts in
app/prompts, rendered with RubyLLM.render_prompt
- Workflows: Correlate multi-agent runs in your telemetry with
RubyLLM.workflow
- Evaluations: Reusable datasets, model criteria, and Ruby assertions with
RubyLLM::Evaluation
- Judgments: Probabilities, choices, and scores with
RubyLLM::Judge and RubyLLM.judge
Individual operations
- Image generation: Create images with
RubyLLM.paint
- Video generation: Create videos with
RubyLLM.animate
- Audio: Transcribe speech with
RubyLLM.transcribe and generate it with RubyLLM.speak
- OCR: Turn documents into markdown with
RubyLLM.ocr
- Embeddings: Generate embeddings with
RubyLLM.embed
- Reranking: Order retrieval candidates by relevance with
RubyLLM.rerank
- Moderation: Content flags, categories, and scores with
RubyLLM.moderate
- Files: Upload once and reuse across chats with
RubyLLM.upload
- Batches: Provider-side batch processing with provider-specific discounts via
RubyLLM.batch
Rails and production
- Rails: Active Record persistence, Active Storage attachments, Hotwire streaming, and generators
- Cost tracking: A per-attempt usage ledger behind
chat.tokens and chat.cost
- Observability: Instrumentation events and OpenTelemetry tracing
- Async: Fiber-based concurrency
- Model registry: Browse capabilities, limits, and pricing across providers
- Providers: OpenAI, Azure, xAI, Anthropic, Gemini, VertexAI, Bedrock, Cohere, DeepSeek, Hetzner, Meta, MiniMax, Mistral, Moonshot AI, Ollama, Ollama Cloud, OpenRouter, Perplexity, GPUStack, ElevenLabs, Deepgram, TypeSafe, Z.ai, and any OpenAI-compatible API
New in 2.2
- Code execution: Let the model run Ruby, Python, JavaScript, and shell in a sandbox on your servers with
with_code_execution
- Skills: Teach the model your workflows with folders of instructions and scripts in the Agent Skills format with
with_skills
- File stores: Keep attachments and long tool results out of the conversation with
with_file_store
- Message queueing: Add messages while a chat runs with
queue
- More providers: Meta, MiniMax, Moonshot AI, and Z.ai
Evaluate your agents
Keep questions and expected answers in a dataset. RubyLLM checks correctness by default. Run the same evaluation from Ruby, RSpec, Minitest, or Rake.
app/agents/support_agent.rb
class SupportAgent < RubyLLM::Agent
instructions "Accept returns of unopened items within 30 days."
end
app/evals/support_evaluation.rb
This is the beginning of the README. Read the rest at the original. Read the full README ↗
