Umeå kommun LLM gateway POC

Website and documentation: https://thekim1.github.io/llmgateway/

One gateway between an organisation's applications and every language model: virtual keys that teams can manage themselves, department/team/key budgets (in USD, or token quotas for on-prem capacity), routing rules that keep personal data on-prem, residency restrictions, provider fallback and retries, request policies (output caps, system prompts) per team and key, prompt caching and a per-key policy for attached files. Applications keep the OpenAI or Anthropic SDK they already use: the gateway translates where a model needs another API (Claude through Chat Completions, tool calls on gpt-5.6/GPT-6 through the Responses API). The gateway stores usage metadata, never prompt or response content. Chat, Responses, Messages, embeddings, speech to text, images and live audio all go through the same pipeline.

.NET 10 data plane and OIDC/BFF control plane, Vue 3/Pinia admin UI, PostgreSQL, authenticated Redis and Aspire 13.6. This is a demonstrator, not a production approval, legal compliance attestation or WCAG certification.

Quick start

Windows with .NET SDK 10.0.401, Aspire 13.6, Docker 28+ and Node 24.15+ (PowerShell):

Set-Location src\admin-ui
npm ci
Set-Location ..\..
.\scripts\Start-Dev.ps1          # with demo data (default)
.\scripts\Start-Dev.ps1 -Empty   # empty database, only the Keycloak login users

Open https://localhost:5173 and sign in as gateway-admin. Details, Visual Studio, the fake LLM and the tests: Local development.

Production, on a Linux Docker host with an OIDC client for the admin UI (Production setup). Up to about 50 messages/s, one host with 2 vCPU, 4 GB RAM and 20 GB disk plus usage data runs everything (Sizing):

curl -fsSLO https://github.com/thekim1/llmgateway/releases/download/vX.Y.Z/ume-llm-gateway-X.Y.Z.tar.gz
tar -xzf ume-llm-gateway-X.Y.Z.tar.gz && cd ume-llm-gateway-X.Y.Z
./deploy/init-deployment.sh        # asks for the OIDC authority and public hostnames
docker compose -f deploy/compose.prod.yaml --env-file deploy/.env up -d --wait

In the admin UI, search (Ctrl K) finds any page or setting by name or topic, and on a phone Menu in the bottom bar opens every page (Admin UI). Usage > Requests and costs with Outcome: All errors lists the failed requests, and every error count links there (Finding failed requests). Settings > Version shows the running version and says when a newer release is out (the admin API checks GitHub; set Admin__CheckForReleases=false without internet access). Model prices for Anthropic, Ollama Cloud, OpenAI and OpenRouter follow the models.dev list, updated daily, which also fills in empty descriptions, capabilities and context windows; turn it off per model or provider to set your own prices (List prices; Admin__PriceListUrl= turns it off). Upgrades: Upgrade guide.

Build and test

Stop the running AppHost before backend or AppHost builds:

aspire stop --apphost src\Ume.LlmGateway.AppHost\Ume.LlmGateway.AppHost.csproj --non-interactive
dotnet build UmeLlmGateway.slnx -v q -nologo
dotnet test --solution UmeLlmGateway.slnx
Set-Location src\admin-ui
npm run lint
npm run typecheck
npm test

The npm projects (src/admin-ui, site, tests/e2e) do not run package install scripts and skip package versions younger than 7 days (npm packages).

Browser tests, benchmarks and the pre-commit hook: Local development and Performance.

Documentation

Every page below is also on the website, built from these files by site/ (see Website).

Get started	Guides	Operate and reference
Introduction	Routing rules	Runbook
Local development	Personal data (PII), Content guard	Upgrade guide
Production setup	Attached files	Data retention and logging
Kubernetes		Updates without downtime
	Speech to text	Security events

Benchmarks: Performance, Streaming findings and the platform comparison.

Changes per release: Changelog. For maintainers (not on the website): Handover, Performance improvement plan, the Demo script and the comparison with agentgateway.