Fast & streaming LLM applications in Rust

Swiftide is an opinionated framework for building LLM applications. It gives you an agent harness, typed task graphs for orchestration, and streaming, composable indexing/query pipelines for RAG.

• Why Swiftide
• Quick Start
• Agent Harness
• Typed Task Graphs
• RAG Pipelines
• Integrations
• Examples
• Project Status
• Contributing
• Core Team Members
• License

Why Swiftide

• Build agents that loop over LLM calls, tool calls, lifecycle hooks, and stop conditions.
• Compose prompt steps, agents, command executors, and domain-specific Rust code in typed task graphs.
• Fan out work into parallel branches and join typed results back into one task output.
• Pause and resume agents or tasks for human approval, external callbacks, or persisted state.
• Bring tools from local Rust functions, custom Tool implementations, or MCP servers.
• Stream large indexing and retrieval workloads through loaders, transformers, embedders, caches, and storage backends.
• Trace agent, task, and pipeline execution with tracing, metrics, and Langfuse support.

The core primitives provide the shared interaction model. Around them, use pipelines for data flows, agents for tool loops, and tasks for graphs of typed hand-offs.

Quick Start

Swiftide keeps default dependencies light. Start with the agent harness and add the integrations your application needs.

cargo add swiftide --features swiftide-agents,openai
cargo add anyhow
cargo add tokio --features macros,rt-multi-thread

If using OpenAI, set the API key expected by the OpenAI-compatible integration:

export OPENAI_API_KEY=...

Agent Harness

Swiftide provides a harnass for building (semi) autonomous agents. The harnass owns message history, call an LLM, invoke tools, run hooks, and stopping. Tools are pure functions and easy to add. The AgentContext abstracts over message history (in memory by default) and provides tools access to the outside world via a ToolExecutor (local by default, see also the docker executor).

use anyhow::Result;
use swiftide::{
    agents,
    chat_completion::{ToolOutput, errors::ToolError},
    traits::AgentContext,
};

#[swiftide::tool(
    description = "Looks up a Swiftide concept",
    param(name = "concept", description = "Concept to explain")
)]
async fn explain_concept(
    _context: &dyn AgentContext,
    concept: &str,
) -> Result<ToolOutput, ToolError> {
    let explanation = match concept {
        "tasks" => "Tasks compose typed nodes into explicit workflows.",
        "agents" => "Agents run LLM completions, tools, hooks, and stop conditions.",
        "pipelines" => "Pipelines stream data through indexing and retrieval steps.",
        _ => "Swiftide composes agents, task graphs, tools, and RAG pipelines.",
    };

    Ok(explanation.into())
}

#[tokio::main]
async fn main() -> Result<()> {
    let openai = swiftide::integrations::openai::OpenAI::builder()
        .default_prompt_model("gpt-4o-mini")
        .build()?;

    agents::Agent::builder()
        .llm(&openai)
        .tools([explain_concept()])
        .on_new_message(|_, message| {
            println!("{message}");
            Box::pin(async { Ok(()) })
        })
        .limit(8)
        .build()?
        .query("Explain Swiftide tasks and agents in one paragraph.")
        .await?;

    Ok(())
}

The agent calls the model, exposes explain_concept as a tool, prints new messages, and stops within the configured turn limit. By default all agents have a tool to stop the loop. This can be customized.

Agent capabilities include:

• function tools through #[swiftide::tool], derived tools, or manual Tool implementations
• lifecycle hooks before and after completions, tools, messages, streaming chunks, start, and stop
• local or custom ToolExecutor implementations for command/file access
• human-in-the-loop approval via ApprovalRequired and feedback-aware contexts
• structured stop and failure payloads with custom JSON schemas
• MCP toolboxes that load tools at runtime
• streaming responses and reasoning-item handling for providers that support them

Start with examples/hello_agents.rs, then look at the human approval, MCP, streaming, resume, and structured-output examples.

Typed Task Graphs

Tasks are Swiftide's orchestration layer. A task is a typed graph of TaskNode steps. Each node has an input type, output type, and error type; transitions decide where the output goes next.

Tasks are strongly typed at build time. It allows you to compose agents, other swiftide components, and functions to create complex automations.

From examples/tasks.rs.

use swiftide::{
    prompt::Prompt,
    tasks::{Task, TaskRunOutcome, Transition},
    traits::SimplePrompt,
};
use std::sync::Arc;

let prompt_model: Arc<dyn SimplePrompt> = Arc::new(openai.clone());
let briefing_agent = BriefingAgent::new(agent);
let mut task: Task<Prompt, String> = Task::new();

let brief = task.register_node(prompt_model.clone());
let decide = task.register_node(briefing_agent);
let render = task.register_node(prompt_model);

task.starts_with(brief);
task.register_transition(brief, move |short_brief| {
    decide.transitions_with(short_brief)
})?;
task.register_transition(decide, move |decision: BriefingDecision| {
    Transition::next(
        &render,
        Prompt::from("Write a hand-off note for {{audience}}: {{summary}}")
            .with_context_value("audience", decision.audience)
            .with_context_value("summary", decision.summary),
    )
})?;
task.register_transition(render, task.transitions_to_finish())?;

match task.run(Prompt::from("Summarize the rollout plan")).await? {
    TaskRunOutcome::Completed(note) => println!("{note}"),
    TaskRunOutcome::Paused => println!("Task paused"),
}

Task capabilities include:

• closure nodes for small glue steps and TaskNode implementations for domain logic
• typed NodeId handles, transitions, and join payloads
• static fan-out with explicit joins
• sequential or parallel branch execution
• pause, resume, and reset support
• adapters for prompt-like Swiftide primitives, chat completions, and tool executors
• TaskAgent for the simple case where an agent should run as a task node

See examples/tasks.rs for a prompt plus custom agent workflow, and examples/tasks_fanout.rs for fan-out and join.

RAG Pipelines

Swiftide includes first-class indexing and querying pipelines for retrieval-augmented generation. Pipelines are streaming and composable: load data, transform it, embed it, cache it, store it, then retrieve and answer with a typed query flow.

use swiftide::{
    indexing::{self, loaders::FileLoader, transformers::{ChunkCode, Embed, MetadataQACode}},
    integrations::qdrant::Qdrant,
};

async fn index(openai: swiftide::integrations::openai::OpenAI) -> anyhow::Result<()> {
    let qdrant = Qdrant::builder()
        .collection_name("swiftide-code")
        .vector_size(1536)
        .batch_size(50)
        .build()?;

    indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"]))
        .with_default_llm_client(openai.clone())
        .then_chunk(ChunkCode::try_for_language_and_chunk_size("rust", 10..2048)?)
        .then(MetadataQACode::default())
        .then_in_batch(Embed::new(openai))
        .then_store_with(qdrant)
        .run()
        .await?;

    Ok(())
}

Indexing supports loaders, caches, chunkers, transformers, batch transformers, embedders, and storage backends. Query pipelines support query transformation, retrieval, response transformation, answer generation, hybrid search, reranking patterns, and evaluation.

Integrations

Swiftide integrations are feature-gated so application builds stay intentional.

Area	Supported integrations
LLM providers	OpenAI and Azure OpenAI, Anthropic, Gemini, OpenRouter, AWS Bedrock Converse API, Groq, Ollama, Dashscope
Tooling	MCP toolboxes, local command execution, custom tool executors
Storage and retrieval	Qdrant, Redis, LanceDB, PgVector, DuckDB, Redb
Loading data	Files, scraping, Fluvio, Kafka, Parquet, executor-backed file streams
Code and text processing	Markdown, text splitting, tree-sitter code chunking and metadata
Observability	tracing, metrics, Langfuse

Examples

The examples has a variety of examples.

• Agents: hello_agents.rs, streaming_agents.rs, agents_with_human_in_the_loop.rs, agents_mcp_tools.rs
• Structured outputs and control tools: stop_with_args_custom_schema.rs, agent_can_fail_custom_schema.rs
• Tasks: tasks.rs, tasks_fanout.rs
• RAG and retrieval: index_codebase.rs, query_pipeline.rs, hybrid_search.rs
• Provider and observability examples: responses_api.rs, responses_api_reasoning.rs, ,

More background is available on the Bosun blog, including posts about tasks, streaming agents, human-in-the-loop flows, and Rust performance for AI tools.

Project Status

Swiftide is pre-1.0. APIs can change while the agent harness and task graph APIs settle around production use. The current API docs and examples are the most reliable source of exact signatures.

Swiftide is part of the bosun.ai project.

Contributing

Contributions are welcome. Read CONTRIBUTING.md, open an issue for design discussion, file a bug report, or propose a feature. Join the Discord for a faster feedback loop.

Before opening a pull request:

1. Run the focused checks for the crates you changed.
2. Run cargo +nightly fmt --all -- --check.
3. Run cargo clippy --workspace --all-targets --all-features -- -D warnings when touching shared behavior.
4. Add or update examples, tests, or rustdoc when behavior changes.

AI-generated code is welcome and should be reviewed like any other code. Keep abstractions small, keep domain logic separate from plumbing, and pay attention to allocations in indexing, querying, and task execution paths.

AI agents can refer to AGENTS.md for workspace layout, commands, and expectations.

Core Team Members

License

Distributed under the MIT License. See LICENSE for more information.