Spanrail

See where every LLM call spends its time.

Self-hosted traces, evals and prompts, plus a gateway that times every call: DNS, connect, TLS, the wait for the first byte, and the transfer.

Website · Architecture · Python SDK

Install · First trace · Gateway · Run it in public · Backups · Limits

curl -fsSL https://raw.githubusercontent.com/Veeky-kumar/SpanRail/main/deploy/install.sh | sh

From the landing page to a first trace. The dashboards in the clip show demo data.

Where did the time go?

Your LLM call took 3 seconds. Was it the DNS lookup? The TLS handshake? The provider thinking? The answer crawling back?

Most tools give you one number. Spanrail's gateway splits each call into its phases and draws them to scale, next to the traces, costs and prompts you already need. Point your client at the gateway, change nothing else, and every call shows up with its own time budget.

It runs on your machine. Spanrail itself never phones home.

What you get

• A time budget for every call. DNS, connect, TLS, upstream wait and transfer on one ruler. Scrub through the average call, see the slow tail (p50 and p95), and see which model eats the bill.
• Traces that nest. Spans and generations with errors, token usage and cost.
• An overview that answers questions. Traces, error rate, latency and spend against the period before, plus a heatmap of when your calls arrive.
• Models side by side. Turn models on and off and see what each one costs and how it behaves.
• Prompts with versions and labels. Fetch production from your app, edit in the dashboard.
• Evals you can rerun. Datasets, runs and deterministic scorers, with a comparison between runs to catch regressions.
• Projects and logins. Each project has its own traces, prompts, datasets and API keys.
• Plain infrastructure. Postgres, Redis, a Next.js app, a worker and an optional Go gateway.

Status

A working MVP, not a hardened product. It has basic rate limits and a switch to close signups, but it does not have password reset, roles with different powers, or an end-to-end test suite. The known limitations list is the honest version.

Install in one command

You need Docker with Compose v2. This pulls the published images, so there is nothing to clone or build:

curl -fsSL https://raw.githubusercontent.com/Veeky-kumar/SpanRail/main/deploy/install.sh | sh

It makes a spanrail/ folder, writes a .env with fresh secrets, starts Postgres, Redis, the web app and the worker (migrations run first), and prints the address: http://localhost:3000. Create an account, make a project, make an API key, then send a trace.

Prefer to read before you run? deploy/install.sh is short. The stack is deploy/docker-compose.yml plus a .env: copy deploy/.env.example and fill in the three secrets with openssl rand -hex 32. Add the gateway with docker compose --profile gateway up -d once your key pair is in .env.

Two things to know:

• The web app listens on 127.0.0.1 only. Put a reverse proxy with TLS in front before you open it up, and set WEB_BIND=0.0.0.0 only behind one. Then read Run it in public.
• Pin a version with SPANRAIL_VERSION=0.1.0 in .env (the default is latest). Back up .env. If you lose API_KEY_SALT, every API key stops working.

Send your first trace

Python (spanrail on PyPI, no dependencies, Python 3.9+):

pip install spanrail

from spanrail import Spanrail

sr = Spanrail("pk-...", "sk-...", base_url="http://localhost:3000")

trace = sr.trace("support-chat", user_id="u_123")
with trace.generation("answer", model="gpt-4o-mini", input="Where is my order?") as gen:
    gen.end(output="It shipped yesterday.", usage={"input": 12, "output": 7})

Events are sent from a background thread and flushed when your program exits. The SDK never raises into your app. More in sdk-python/README.md.

TypeScript (not on npm yet, so import the local source):

import { Spanrail } from "./sdk"; // local source for now

const spanrail = new Spanrail({
  publicKey: process.env.SPANRAIL_PUBLIC_KEY!,
  secretKey: process.env.SPANRAIL_SECRET_KEY!,
  baseUrl: "http://localhost:3000",
});

const trace = spanrail.trace({ name: "my-agent-run", userId: "user_123" });
const span = trace.span({ name: "tool:search", input: { query: "..." } });
span.end({ output: { results: [] } });

const generation = trace.generation({ name: "llm:answer", model: "gpt-4o-mini" });
generation.end({ output: "...", usageDetails: { input: 120, output: 40 }, costDetails: { input: 0.00006, output: 0.00004 } });

await spanrail.shutdown(); // flushes queued events before the process exits

Fuller examples you can run:

• examples/simple-agent.ts is a multi-span agent with a deliberate tool failure, so you can see errors and nesting.
• examples/run-dataset-eval.ts runs the whole loop: dataset, run, scorers.

This is the beginning of the README. Read the rest at the original. Read the full README ↗