Halro

Halro is a single-binary, security-first LLM gateway. It gives applications one governed API for multiple model providers while keeping upstream credentials, budgets, routing, redaction, audit, and usage accounting inside a locally owned control boundary.

• One Go binary with an embedded Chinese/English Admin console
• No external database, cache, CDN, or browser-side secret storage
• OpenAI-compatible Chat, Embeddings, and Stateless Responses
• Anthropic-compatible Messages with portable and native routing
• Versioned, capability-aware Provider Profiles and fail-closed transport
• Durable accounting, audit integrity, anomaly containment, and redaction
• Repository-supported, manually operated Primary/Replica HA (production admission and target-environment validation are still pending)
• Authenticated Prometheus metrics, Alertmanager rules, and an independent dead-man monitor

[!IMPORTANT] Chat, Embeddings, Stateless Responses, and Anthropic Messages have published compatibility contracts. Phase 2 media/resource endpoints are Experimental, as is GET /v1/models, which lists the aliases a Gateway Key may name. Realtime WebSocket/WebRTC and stateful Responses are not implemented.

Quick start

Requirements

• Go 1.26.10 or later
• Node.js/npm only when rebuilding the embedded Admin console

Start a new loopback-only local instance:

make start

On first run, Halro creates config.yaml, initializes encrypted local storage, and prints the Admin URL. Open /admin/setup to create the first administrator, then configure this path in the console:

Credential → Provider → Deployment → Route → Project → Gateway Key

The Gateway Key is displayed once. Applications use that gw_... key and a public model alias; they never receive the Provider credential or upstream model identifier.

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer $HALRO_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "chat",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

For SDK examples and the complete Admin workflow, read the User Guide, also available in 简体中文. For deployment, upgrades, backup, recovery, and hardening, use the Operator Guide. For a three-member HA deployment and manual handoff, read the HA Usage Guide, also available in 简体中文. The Docker example below uses v0.8.5, which does not include the cluster commands.

Run with Docker

Every tag publishes multi-architecture images (linux/amd64, linux/arm64):

docker pull ghcr.io/akz142857/halro:v0.10.1
docker pull ghcr.io/akz142857/halro-deadman:v0.10.1   # the independent watchdog, deployed separately

latest follows the newest tag; pin by the manifest digest reported by the release for anything you deploy. For example, this is the previous v0.7.1 manifest digest:

ghcr.io/akz142857/halro@sha256:aaa3bfa22ea0bb032bddccacd2ed2f4fda978165db14d5ce3198601f51e29f91

The runtime is distroless with no shell, runs as uid 65532, and its ENTRYPOINT is the halro binary. The default command is serve --config /etc/halro/config.yaml, and serve never initializes storage, so the first run is an explicit init.

Configuration is never merged with built-in defaults — a partial file is rejected key by key — so start from the complete annotated configs/config.example.yaml, or from the config.yaml that a local make start writes, and change the three values whose defaults are wrong for a container:

server:
  # Container loopback would refuse every request arriving through -p.
  gateway_listen: "0.0.0.0:8080"
storage:
  # A child of the mount; the mount point itself must not be the data directory.
  data_dir: "/var/lib/halro/data"
  master_key:
    file: "/run/secrets/halro-master.key"

Initialize once, with /run/secrets writable so init can create the file-mode Master Key at 0600:

mkdir -p ./halro-secrets
sudo chown 65532:65532 ./halro-secrets && chmod 700 ./halro-secrets
docker volume create halro-data

docker run --rm --user 65532:65532 \
  -v "$PWD/config.yaml:/etc/halro/config.yaml:ro" \
  -v "$PWD/halro-secrets:/run/secrets" \
  -v halro-data:/var/lib/halro \
  ghcr.io/akz142857/halro:v0.10.1 init --config /etc/halro/config.yaml

Then serve, publishing only the Gateway on host loopback:

docker run -d --name halro --user 65532:65532 \
  -v "$PWD/config.yaml:/etc/halro/config.yaml:ro" \
  -v "$PWD/halro-secrets:/run/secrets:ro" \
  -v halro-data:/var/lib/halro \
  -p 127.0.0.1:8080:8080 \
  ghcr.io/akz142857/halro:v0.10.1 serve --config /etc/halro/config.yaml \
    -allow-insecure-public-listen

-allow-insecure-public-listen covers Gateway plaintext only and exists because of the 0.0.0.0 bind above — neither alone is enough, and together they describe a host-local development boundary, not a public one. Admin and Metrics stay on container loopback in this shape; publish them only after enabling TLS and mounting the certificate, key, and Metrics client CA.

Halro has no built-in certificate issuance: tls.certificates names keypairs it loads, and obtaining or renewing them belongs to whatever already does that here — certbot, an internal CA, or a reverse proxy such as Caddy that terminates TLS in front of a loopback-bound Halro. Replacing the files and sending SIGHUP swaps the certificate without a restart. Both shapes, and which settings each one requires, are in TLS and inbound exposure.

Two container facts that bite:

• One process per data directory. For the Standalone Docker example here, mount the persistent parent at /var/lib/halro; use Kubernetes replicas: 1 with a Recreate strategy. HA is a separate Primary/Replica deployment with a distinct data directory and PVC for each member, approved seeding, and manual promotion; see the HA Usage Guide.
• healthy is not reachability. HEALTHCHECK calls a readiness URL from inside the container, so it proves the process is ready, not that a published port, certificate name, firewall, or reverse proxy works. Probe the external address separately, and set HALRO_HEALTH_URL to an HTTPS name the mounted certificate covers once TLS is on.

Registry images carry no cosign signature or registry attestation. The signed and attested artifacts are the release archives, so when you need that provenance chain verify halro-container-<arch>.tar.gz as described in Verify release downloads and load it directly instead of pulling:

gzip -dc halro-container-amd64.tar.gz | docker load

Full container guidance — TLS, key custody, the backup and restore layout, and the Kubernetes manifest — is in the Operator Guide.

Data durability and encrypted backup

Container or Pod replacement is safe only when the complete storage.data_dir remains on persistent storage. In Standalone mode, run one instance for that directory and use Kubernetes replicas: 1 with a Recreate strategy. In HA mode, each member owns its own persistent directory and no two members may mount the same writable PVC.

File-mode master.key is not stored in the encrypted backup. Keep it outside the data directory and back it up independently from both the .hmbk archive and its dedicated 32-byte backup key. Never restore only halro.db or mix the database, Ledger, Audit, Usage, and Provider-object files from different snapshots.

Standalone backups are deliberately offline: stop Halro, create the archive, verify it, and regularly perform an isolated restore drill. HA has a separate Replica maintenance, backup, and report procedure in the HA Usage Guide. For containers, mount the persistent parent directory and configure storage.data_dir as its child so restore can atomically rename the data directory on the same filesystem.

With Halro stopped, the repository helper creates and immediately verifies an encrypted backup:

make backup

It defaults to config.yaml, backups/, and the git-ignored backup.key. Production operators should provide independent locations explicitly:

make backup \
  CONFIG=/etc/halro/config.yaml \
  BACKUP_DIR=/secure-backups \
  BACKUP_KEY_FILE=/secure-secrets/halro-backup.key

The helper generates the Backup Key with mode 0600 when it is absent, runs offline doctor, creates the archive, and runs backup verify. Move or escrow the key independently; deleting it makes every archive encrypted with it unrecoverable.

See Encrypted backup and restore for Docker/Kubernetes layouts, upgrade sequencing, key custody, retention, and recovery commands.

Verify release downloads

Verify the signed checksum manifest before trusting it, use that manifest to check all downloaded bytes, and then verify every artifact bundle against the exact release workflow identity:

Requires cosign v2.2 or newer (--bundle reads the new Sigstore bundle format).

# Download everything: checksums.txt lists every published artifact, and both
# the checksum check and the loop below expect the files to be present.
gh release download v0.10.1 --repo akz142857/Halro   # or download all assets by hand

# Official releases are signed by the release workflow dispatched from main.
# The certificate identity therefore names refs/heads/main; the checksums and
# GitHub attestation below bind each artifact to the exact release commit.
COSIGN_IDENTITY='https://github.com/akz142857/Halro/.github/workflows/release.yml@refs/heads/main'
COSIGN_ISSUER='https://token.actions.githubusercontent.com'
cosign verify-blob \
  --certificate-identity "$COSIGN_IDENTITY" \
  --certificate-oidc-issuer "$COSIGN_ISSUER" \
  --bundle checksums.txt.sigstore.json checksums.txt
# --ignore-missing lets you verify only the platform you downloaded; drop it to
# require every listed file. Without either, a partial download exits non-zero.
sha256sum --check --ignore-missing checksums.txt  # macOS: shasum -a 256 --check --ignore-missing checksums.txt
for artifact in halro-* halro.spdx.json; do
  case "$artifact" in *.sigstore.json) continue ;; esac
  [ -f "$artifact.sigstore.json" ] || continue
  cosign verify-blob \
    --certificate-identity "$COSIGN_IDENTITY" \
    --certificate-oidc-issuer "$COSIGN_ISSUER" \
    --bundle "$artifact.sigstore.json" "$artifact"
done

Do not verify the blobs against an unsigned checksums.txt, and do not replace the exact tag identity with a branch identity — substitute the tag you actually downloaded. Release candidates use their own exact refs/tags/vX.Y.Z-rc.N identity.

API status

API	Status	Notes
POST /v1/chat/completions	Compatible	JSON and SSE; OpenAI Go/Node/Python SDK matrix
POST /v1/embeddings	Compatible	Per-Profile coverage applies: OpenAI, Azure OpenAI, Gemini Beta, and the OpenAI-compatible Profile
GET /v1/models, GET /v1/models/{id}	Experimental	The aliases the key's Project may name and a Route serves; answered from Halro's own configuration, no upstream call, owned_by is always halro. An id is an alias an operator may repoint, so listing one promises only that it will not answer model_not_found
POST /v1/responses	Compatible subset	Stateless Create and text SSE, plus background: true deferred submission; store:true and stateful fields are rejected
GET/POST .../cancel/DELETE /v1/responses/{id}	Experimental	Deferred retrieval, cancel, and delete for background: true; per-Project opt-in, answers sealed and retained at most 24 h (ADR 0024)
POST /v1/messages	Compatible	Anthropic JSON/SSE; portable or exact native Profile routing
POST /v1/messages/count_tokens	Compatible	Anthropic token counting over the same Profile routing
Moderations, Images, Audio	Experimental	Strict Phase 2 endpoint and Provider Profile contracts
Files and Batches	Experimental	Project-scoped opaque IDs, idempotency, ownership, private local objects
/v1/rerank and Async Invoke	Not served in this build	Halro extensions whose only backing Profiles — Cohere Rerank 3.5 on Bedrock Agent Runtime, Nova Reel Async on Bedrock Runtime — are withheld, so no Deployment can be created for either
Realtime WebSocket/WebRTC	Not implemented	Architecture only; no Realtime data plane is claimed

The authoritative machine-readable contract is docs/compatibility/endpoint-manifests.json. Endpoint maturity does not promote every attached Provider Profile: each profile_coverage entry carries its own status and field-level deviations.

Providers

Every upstream is bound to an immutable, versioned Profile that fixes its Access Surface, operations, credential scheme, and capability evidence.

Provider	Current scope
OpenAI	Chat, streaming, embeddings, Stateless Responses, and Experimental Phase 2 media/resources
Anthropic	Native Messages JSON/SSE, count_tokens, tools, and signed Thinking round-trip; no embeddings
Azure OpenAI	Deployment-scoped Chat/stream/embeddings with an explicitly pinned API version
DeepSeek	OpenAI-compatible Chat/stream with reasoning and cache-tier usage; no embeddings
OpenAI-compatible	Chat/stream/embeddings with explicitly declared optional capabilities
MiniMax	One host and one key across three Profiles: Chat, Stateless Responses (no streaming), and Anthropic Messages
Kimi	One key across the connection group: Chat (the only face reaching all published models) and Anthropic Messages; its Responses Profile is withheld
Gemini Beta	Native text generation/SSE and float embeddings translation
AWS Bedrock Mantle Beta	Isolated OpenAI Chat, Stateless Responses, and Anthropic Messages Profiles

Bedrock is offered through Mantle alone. The five Bedrock Runtime and Agent Runtime Profiles — Converse, Titan Text Embeddings V2, Titan Image V2, Cohere Rerank 3.5, and Nova Reel Async — are withheld in this build: the implementation is present, but the served Profile matrix omits them and every write path refuses them, so no Deployment can be created against one. An install that already holds such a connection still starts and can delete it.

Provider capabilities and per-field coverage are checked before Provider I/O. Unknown or unsupported fields are rejected instead of silently dropped. Runtime, Agent Runtime, and Mantle credentials, audiences, quotas, concurrency, and capability evidence are not merged.

Security and correctness boundaries

• Provider credentials are encrypted with AES-256-GCM and audience-bound AAD.
• Gateway Keys are 256-bit random values stored only as SHA-256 representations.
• Outbound transport is HTTPS-only, allowlisted, DNS/IP validated, dial-pinned, redirect-free, and independent of environment proxies.
• Projects enforce allowed routes, budget reservations, RPM/TPM/concurrency, CIDR rules, redaction policy, and Token Guard policy before upstream access.
• Accounting uses a framed, checksummed WAL with conservative unknown-outcome settlement; retries never assume an ambiguous Provider request was absent.
• Files, Batches, and Async resources keep project-scoped owner mappings pinned to Provider, Deployment, Profile, and Region.
• Admin secrets and CSRF state are never persisted in browser storage; the production bundle is scanned for secret canaries and persistence APIs.
• Prompts and response bodies never reach a log, a metric, or an audit record. Two opt-in stores are the only places caller-written content is kept at rest, and both change what the data directory contains, so enabling either is a decision rather than a default: gateway.failure_capture retains the Gateway request body, normalized request and upstream answer or transport error of a failed call, and the per-Project deferred tier (background: true) retains the answer of a successful one. Both are sealed under the Master Key, bound to their request and Project, bounded in size and count, and swept on expiry. A captured failure payload is readable only through an audited admin action; a deferred answer is readable only by the Project that submitted it, within its retention window.

See the Threat Model, Gateway correctness contract, and Security Policy for the complete boundary.

Architecture

Client / SDK
    │  Gateway Key + public model alias
    ▼
Protocol Facade ──► Semantic operation + governance requirements
    │
    ▼
Auth / policy / budget / redaction / accounting
    │
    ▼
Capability-aware router ──► versioned Provider Primitive
    │
    ▼
SafeTransport ──► OpenAI / Anthropic / Azure / DeepSeek / Gemini / Bedrock

Without a replication configuration block, Halro runs as a Standalone, single-writer system backed by bbolt metadata, an authoritative Ledger WAL, private local objects, and Parquet usage partitions. The current source tree also implements manually operated Primary/Replica HA. Its repository gates and local exercise do not constitute production admission: target-environment fault injection, soak, RTO/RPO evidence, and the release gates are still required. Realtime remains unimplemented.

Observability

Halro exposes an authenticated Prometheus-format Metrics endpoint. The repository includes a versioned single-host Core reference deployment made up of Prometheus and Alertmanager, together with recording rules, alert rules, rule tests, configuration validation, and an isolated runtime smoke test. Prometheus is the metrics and alert authority.

The halro-deadman binary is deployed in a separate failure domain. It checks Halro, Prometheus, and Alertmanager readiness, verifies Prometheus sample freshness, and sends durable heartbeat and down/up events to an independent receiver.

For HA clusters, the release archive also includes halro-ha-health, an independent read-only health view for cluster, member, replication, and client-Service evidence. Its deployment contract requires per-member collection, a controlled operator entry point, and target-environment acceptance before production use.

Validate the repository-provided configuration before deployment:

make observability-check

Secret preparation, Linux and macOS Compose commands, the independent dead-man contract, and production boundaries are documented in the Prometheus/Alertmanager deployment guide.

Documentation

Use and operations

• User Guide
• Operator Guide
• HA Usage Guide — deployment, status, manual handoff, recovery, and validation
• Backup and restore
• Deferred responses — background: true
• Choosing an AWS access surface
• Metrics reference
• Prometheus/Alertmanager deployment
• Observability operations runbook
• Webhook payloads

Contracts and implementation evidence

• Implementation status
• Endpoint compatibility manifests
• OpenAI compatibility contract
• Provider capability contract
• Gateway idempotency contract
• Crash recovery matrix
• Provider real-test matrix

Architecture and governance

• 多协议 LLM API、Provider 与 Realtime 架构设计
• Distributed evolution and state ownership
• Architecture Decision Records
• Threat Model
• Audit integrity

Development

Run the local quality gates:

go test ./...
go test -race ./...
go vet ./...

The Admin console supports standard TOTP two-factor authentication with multiple independently revocable authenticators and one-time recovery codes. It works with Microsoft Authenticator, Google Authenticator, 1Password, and other compatible apps without contacting those vendors.

Rebuild and verify the embedded Admin console:

cd web
npm ci --ignore-scripts
npm run typecheck
npm test -- --run
npm run build
cd ..
go build -trimpath -o bin/halro ./cmd/halro

Real Provider smoke tests are opt-in and may be billable. They require isolated test credentials, explicit environment flags, and hard account budgets; see the Provider real-test matrix.

Read CONTRIBUTING.md before submitting changes. Release process and evidence gates are documented in docs/guides/releasing.md.

Community and license

Use GitHub Issues for bug reports and feature proposals. Security vulnerabilities must follow SECURITY.md and must not be posted in a public issue. Community expectations are in CODE_OF_CONDUCT.md, and support guidance is in SUPPORT.md.

Halro is licensed under the Apache License 2.0. Third-party attributions are documented in NOTICE and THIRD_PARTY_NOTICES.md.