theloop

---

The request carries everything: provider, model, key, prompt, full history, tools (URL + schema), limits. theloop runs "model → tools → model", streams text over SSE, delivers the result and billing-grade usage by callback — and forgets the run.

Built for teams that ship assistants to many users: multi-tenant by construction, per-request keys, one usage record per provider attempt, write tools as intents that your system approves and executes.

[!NOTE] Status: v1 — the contract is frozen (SPEC.md) and implemented: Node 22, TypeScript, Hono, Vercel AI SDK 6. Tests run without network or keys. New features arrive as additive fields, like skills and deferred tools (CHANGELOG.md). "1.0" is a promise about the contract (semver: no breaking changes within 1.x), not a claim of operational maturity — the implementation is young; expect fixes, not contract changes.

One run, the whole story

With the engine running (Quickstart), one synchronous run (wait: true), with the Anthropic key taken from ANTHROPIC_API_KEY:

curl -s http://localhost:8080/runs \
  -H "Authorization: Bearer $ENGINE_TOKEN" -H 'Content-Type: application/json' \
  -d '{ "contract_version": "1",
        "llm": { "provider": "anthropic", "model": "claude-sonnet-5" },
        "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Hello!" } ] } ],
        "wait": true }'

The answer is a synchronous 200 with the result body, usage included. No database was harmed. Without wait, the response carries stream_url (SSE) and cancel_url, and the result goes to callback.url. Tools, write intents, limits and everything else — SPEC.md.

Why

• Everything in the request, nothing in the engine. No database, no configs, no tenant registry. Your backend decides what the assistant is on every call.
• One result, at-least-once. Callback with retries, deduplication by run_id, results readable via GET /runs/{id} for recovery, late results placed by history revision.
• Writes are intents. A tool with effect: "write" answers deferred with an operation id; the mutation happens in your system after approval — never inside a tool call. With tools built that way, a prompt-injected model can request an action but cannot carry it out — see Trust model for what that depends on.
• Usage you can bill. Every provider attempt, including failed and interrupted ones, with uncached input, cache read / write and an explicit estimated flag. Optional cost from your price list, with token and cost budgets.
• Controlled egress. HTTPS only, host allowlist, no private IPs, no redirects, header filtering; provider keys never in logs, SSE, callbacks or error bodies.
• Providers. Anthropic, OpenAI, DeepSeek, GLM (z.ai), GigaChat, MiniMax, Kimi and any OpenAI-compatible endpoint, with per-key queues and AIMD rate limiting.
• Fallback chain. Send llm as an array, and a provider failure moves the run to the next model instead of failing it. The result says which model answered and why the others were skipped.
• Skills and deferred tools. skills[] in the SKILL.md shape: the model sees an index and loads a skill's instructions and tools mid-run. Deferred tools stay out of the context until a skill or search_tools (BM25) brings them in, and the prompt cache survives (tool_reference on Anthropic). Still stateless: what is loaded comes from the history. See Skills and deferred tools.
• Debuggable. An event journal per run and session (GET /logs), with content, without shipping prompts to your log pipeline: in memory, and with LOG_DIR in day files on disk that survive a deploy and keep a week.
• Presentation invariants. Mark what the harness renders, and the engine keeps one card or deck per run, keeps blocks out of tool steps, and checks that card values came from tool results.
• Always answers on a limit. The last allowed step goes without tools, so a run that hits or still ends with an answer, not an empty error.

Skills and deferred tools

A request can carry instructions and tools the model pulls in only when it needs them:

"tools": [
  { "name": "getOrder", "...": "...", "effect": "read" },
  { "name": "createReturn", "...": "...", "effect": "write", "defer_loading": true }
],
"skills": [
  { "name": "order-returns",
    "description": "Return and refund policy. Use when the user asks about a return.",
    "body": "# Returns\n\nCheck the order status first ...",
    "tools": ["createReturn"] }
]

The model sees getOrder, a built-in load_skill and a one-line index of skills in the system prompt. When the conversation turns to returns, it calls load_skill("order-returns"), gets the instructions, and createReturn is callable from the next step. A deferred tool that belongs to no skill is found with the built-in search_tools. Built-ins cost a step but not a tool call, and the harness only has to store the engine's tool messages as usual: the next run restores what was loaded from the history. Details: SPEC §6.8, RFC 0001.

Quickstart

From source (Node 22.9+):

git clone https://github.com/paveler/theloop.git && cd theloop
npm install
cp .env.example .env      # set ENGINE_TOKEN and at least one provider key
npm run dev               # reads .env; http://localhost:8080

Docker, without cloning — images are published on every release:

docker run -p 8080:8080 -e ENGINE_TOKEN=change-me -e ANTHROPIC_API_KEY=sk-ant-… ghcr.io/paveler/theloop:1

Or build your own: docker build -t theloop . && docker run -p 8080:8080 --env-file .env theloop.

Then send the run from One run, the whole story.

This is the beginning of the README. Read the rest at the original. Read the full README ↗