TheYgent

TheYgent is a no-code, local-first AI orchestration platform for building and running AI agents. You compose an agent visually — LLM steps, tools, MCP servers, retrieval, routers, guardrails, human approval — and run it against models on your infrastructure: llama.cpp anywhere, MLX on Apple Silicon, vLLM on NVIDIA GPUs, or any OpenAI-compatible API you register. Local and hosted models mix freely in one graph, chosen per node and per call by a logical id.

The promise is sovereignty and anti-lock-in, stated honestly: TheYgent is never an involuntary middleman between you and your models. Prompts, payloads, and weights are never routed through anyone's servers — when something crosses a wire, it does so because you sent it there.

What's in the box

• Visual builder — drag-and-drop canvas; the graph is the agent. Publishing mints an immutable, content-addressed version you can roll back to.

• Built-in local inference — TheYgent runs the models itself: install one from the in-app Hugging Face catalog (each variant shows whether it fits your RAM), and llama.cpp, MLX on Apple Silicon, or vLLM (experimental) on NVIDIA starts on the first call and unloads when idle — no separate model server to run. Chat, vision, embeddings, speech-to-text, text-to-speech, and image generation, served locally.

• Multi-engine, one id — one logical model id per call; swap the engine underneath (Apple Silicon ↔ GPU ↔ hosted API) with a single registration change and every call site untouched. Local and hosted models mix freely, per node.

• Private by default — no telemetry, usage analytics, update checks, or vendor account, and it runs offline or air-gapped once installed. Every connection it makes on its own is listed, with when and why.

• Tools & MCP — built-in tools, HTTP tools, and any MCP server over stdio or HTTP, including servers generated in-process from an OpenAPI or GraphQL spec; a registry to browse and install from, with full auth including interactive OAuth.

• RAG — crawl a site or upload documents, embed locally into pgvector, wire a retrieval node into any graph.

• Durable runs and human-in-the-loop — checkpoint every step; kill the process mid-run and it resumes from the last completed node. Human-approval gates that wait for days and survive restarts, plus subgraphs, loops, and fan-out maps.

• Observability — a zoomable span waterfall for every run (per-node inputs, outputs, timing, token usage) stored in your Postgres, with opt-in OTLP export to your own OpenTelemetry collector.

• Automation — deploy a saved agent behind a token-authed endpoint, an HMAC-signed webhook, or a cron schedule.

• Chat & bench — a unified chat for any model or agent (thinking view, voice, vision), and a bench for testing and comparing models and agents side by side.

• AI gateway for Claude Code — put your team's Claude Code behind a proxy that cuts what it costs: it records what every exchange cost and what routing would save, then — when you choose — routes work to cheaper Claude models and, in shares you set and measure first, to open models you run yourself. Per-session timelines explain every decision.

• An MCP server for coding agents — connect Claude Code, Codex or OpenCode and they can offload work to your local models, run and build TheYgent agents, and index and search your docs and code (make mcp-setup prints the setup).

Architecture at a glance

Two planes, four process types, one topology everywhere:

flowchart LR
  UI["interface — React SPA :5174"]
  API["control plane — FastAPI :8080\nruns · agents · sessions · triggers\nMCP · RAG · observability"]
  W["worker — durable runtime (opt-in)"]
  PG[("Postgres + pgvector")]
  subgraph ip["inference plane :8081 — user-controlled"]
    GW["OpenAI-compatible gateway /v1/*\nengine manager /admin/*"]
    ENG["llama.cpp · MLX · vLLM"]
  end
  HOSTED["any OpenAI-compatible API"]
  AG["AI gateway :8082\nClaude Code → Anthropic or local models"]

  UI --> API
  AG -->|"records (metadata only)"| API
  AG -->|"local models"| GW
  UI -->|"raw prompts & media, direct"| GW
  API -->|"logical model ids over HTTP"| GW
  W --> GW
  API --- PG
  W --- PG
  GW --> ENG
  GW -->|passthrough| HOSTED

• The inference plane is user-controlled and holds everything heavy: engine processes, weights, the model registry, credentials. The control plane orchestrates: the graph runtime, versioned agent registry, sessions, triggers, MCP hosting, RAG, secrets, observability. The control plane never imports an engine — it reaches inference only over OpenAI-compatible HTTP with logical model ids.
• Agents are documents in a typed IR with a content hash over the canonical graph (layout is never hashed). The same IR runs on an in-process interactive runtime and on a journaled durable runtime.
• The AI gateway sits between Claude Code and its models in the same trust domain: it forwards traffic byte for byte, changes only the model when it routes, reaches local models through the inference plane, and sends the control plane metadata only.
• The full map, with the reasoning behind each boundary, is in docs/dev-docs/architecture.md.

Developer documentation

The full contributor docs live in docs/dev-docs:

Page	Covers
Architecture	The two-plane split, the three seams, process topology, design principles
Control plane	The FastAPI orchestration service: runs, agents, sessions, triggers, MCP, RAG, observability, secrets, settings
Graph execution	How an IR graph runs: the walker, the node set, $in substitution, conditional dataflow
Durable execution	Crash-safe runs: the journaled workflow, durable-only nodes, the worker process
Inference plane	Engines and the gateway: bindings, modalities, lifecycle, catalog, credentials
AI gateway	The Claude Code proxy: pass-through, identity, the routing policy, local models and trials, records, the admin surface
MCP server	TheYgent as tools for coding agents: the tools, auth with personal API keys, local-file indexing
Interface	The React SPA: the canvas/IR adapter, generated types, chat, bench, the run waterfall
IR and packages	The IR document envelope, contentHash, validation, TS codegen, the gateway client

Contributor rules — what must hold before a change lands — are in AGENTS.md, which also maps every module's own AGENTS.md (what it does, how it connects, a file map, local rules).

Getting started (development)

cp .env.example .env   # set DATABASE_URL (a reachable Postgres)
make engines           # local engine servers (macOS): llama.cpp, whisper.cpp, ffmpeg,
                       # mlx-lm, mlx-vlm, mlx-audio — chat, vision, embeddings,
                       # speech-to-text, text-to-speech
make up                # install deps, migrate, start inference-plane (8081) +
                       # control-plane (8080) + interface (5174) + AI gateway (8082)

make help lists everything else (status, logs, restart, test, lint). /readyz on the inference plane reports which (engine, modality) slots this host can actually run.

Running with Docker (the second way)

The same stack, containerized — Postgres (pgvector) included, same ports:

make docker-up         # postgres + migrations + inference-plane + control-plane + interface
make docker-down       # stop (data volumes survive; `docker compose down -v` drops them)

The containerized inference plane bundles a CPU llama.cpp build (chat, embeddings, vision). MLX is Apple-Silicon-bare-metal only — to keep local MLX inference, run the inference plane on the host and overlay docker-compose.host-inference.yml (see that file's header). The standalone durable worker is opt-in: THEYGENT_DURABLE=1 docker compose --profile worker up -d; so is the AI gateway, which needs a service key an admin mints first: docker compose --profile gateway up -d. JS-rendered crawling for retrieval sources (render_js) needs a browser baked into the control-plane image — opt in with WITH_JS_RENDER=1 docker compose build control-plane (~500MB); without it, static fetch still works and a render_js ingest fails with the rebuild hint.

Kubernetes (deploy/k8s)

Dev flow against minikube — images built locally, side-loaded, applied with kustomize:

make k8s-load          # docker compose build + minikube image load
make k8s-apply         # kubectl apply -k deploy/k8s
kubectl -n theygent port-forward svc/interface 5174:80       # then :8080 / :8081 / :8082 likewise

The control-plane, worker and AI gateway carry HorizontalPodAutoscalers (scale up/down with load); the AI gateway runs as a StatefulSet and starts once its Secret exists (see deploy/k8s/ai-gateway.yaml); deploy/tests/ guards the Dockerfiles, compose files, and manifests in CI.

Contributing

CONTRIBUTING.md covers the mechanics — development setup, pull requests, CI, and the contributor license agreement (CONTRIBUTOR_LICENSE_AGREEMENT.md). The rules every change must satisfy are in AGENTS.md: all tests pass (make test + make lint), user- and developer-docs move with behavior changes, and the architectural guardrails hold. Security reports: see SECURITY.md.

License

Fair-code, under the Sustainable Use License: free to use, modify, self-host, and build on for your own internal business or personal purposes; commercializing TheYgent itself — selling it, or offering it as a hosted service — is reserved. See LICENSE.md.