Gavio

The AI request runtime and inspector for production systems.

PII protection · audit trails · runtime events · reliability · cost control — as composable interceptors. Same API in Python, Java, and JavaScript.

📖 Docs: manojmallick.github.io/gavio

---

What is Gavio?

Gavio sits between your application and any LLM provider. Every request passes through a pre/post interceptor chain — PII redaction, retries, cost tracking, audit logging, policy packs, tool runtime, runtime context — before and after the provider call:

Request → [ PII Guard · Secret Scanner · … ] → Provider → [ … · PII Restore · Audit ] → Response

Every team re-implements the same production concerns around LLM calls: redact PII before it leaves the building, retry on 429s, fall back to a second provider, log an audit trail, track spend. Gavio ships them once, as swappable interceptors, with identical behaviour across three languages — enforced by shared test vectors.

• Provider-agnostic — OpenAI, Anthropic, Gemini, Azure, OpenRouter, Ollama, Mock. Switching is a config change.
• Zero mandatory dependencies in every core (stdlib HTTP everywhere — no vendor SDKs).
• Dev mode — the whole stack runs in-process with a mock provider. No API key, no network.
• Audit by default — every call logged as metadata + SHA-256 content hashes (never raw text).
• Runtime event export — metadata-safe JSONL events for integrations with gateways, observability, and eval workflows.
• Ecosystem integrations — compatibility matrix, dependency-light metadata helpers, adapter payloads, generated conformance evidence, and sample production apps for LiteLLM, promptfoo, Langfuse, OpenLIT, LangChain, LangGraph, Vercel AI SDK, and related stack tools.
• Self-hosted Control Plane — optional local/private server and admin UI for runtime config, policy rollout, budget config, audit search, workflow releases, and cached SDK fallback.
• Production Trust Package — metadata-only release evidence bundles that verify audit-chain, runtime-event, policy, benchmark, and document evidence.
• Platform Runtime Profile — metadata-only readiness score and gap report for platform-grade production runtime posture.
• Platform Workflow Release — one metadata-only release artifact for prompts, evals, policy packs, trust bundles, and runtime profiles.
• OTel bridge — runtime events map to OpenTelemetry-style spans for production APM pipelines.
• Inspector — opt-in dev-time visualizer: live traces, per-interceptor waterfall, PII redaction diffs, and pipeline lints at http://127.0.0.1:7411 (inspect(true) or GAVIO_INSPECT=1).
• Inspector agentic & production mode — multi-agent call graphs and session views, trace replay & edit-resend (full mode only), RED stats, hash-chain verification, PII-sanitized export of any trace as a test case, and a read-only dashboard over a persisted audit store: gavio inspect --store audit.jsonl.

Status: v3.2.1 is the current stable package line. Gavio has an API stability guarantee, a 24-month 1.x LTS policy, and release automation that checks lockstep SDK versions before publishing. See STABILITY.md and the CHANGELOG.

---

Version map

The package line stays normal semver: v0.11.0 through v0.14.0 completed the last pre-1.0 product milestones, then v1.0.0 became the stable release.

Version	Focus	Main shipped surface
0.11.0	Cost Intelligence	Tenant/feature/user attribution, cost reports, retry overhead, cache savings, scoped budget fallback
0.12.0	Policy Pack architecture	Core and FinTech policy-pack manifests plus custom regex-rule packs
0.13.0	Adapter & positioning	OpenRouter adapter, first-class runtime context, AI Request Runtime / Inspector positioning
0.14.0	Tool Runtime	Tool schema validation, freshness checks, conflict detection, confidence, provenance
1.0.0	Stable release gate	Lockstep SDK versions, API stability, 24-month 1.x LTS policy, release hygiene checks
1.1.0	Positioning + Integration Foundation	Metadata-safe runtime event/export contract, JSONL exporters, integration docs, runtime export examples
1.2.0	Cost Governance v2	Budget policy/decision contracts, budget stores, fallback/downgrade/dry-run decisions, budget-aware reports, CLI report
1.3.0	Observability + OTel	OpenTelemetry-style span exporter, shared OTel vectors, JSONL-to-OTel conversion
1.4.0	Prompt Registry + Evals	Versioned prompt templates, metadata-only lineage, deterministic eval suites and privacy-safe reports
1.5.0	Tool Runtime v2	Registry-backed permissions, approval gates, replay records, MCP metadata capture
1.6.0	Policy Pack Catalog	Signed domain policy-pack manifests, catalog loaders, overrides, suppression rules, and domain examples
1.7.0	Self-hosted Control Plane	Local/private runtime config, policy rollout, budget config, audit search, snapshots, and SDK cache fallback
1.8.0	Production Trust Package	Metadata-only release evidence bundles, deterministic verification, threat model, benchmark and reference-architecture docs
1.9.0	Ecosystem + Integrations	Integration catalog helpers, shared compatibility vector/schema, common AI stack recipes, and full-stack integration smoke example

---

Architecture

Gavio is a thin core (Gateway + InterceptorChain + the request/response model) that everything else plugs into. A request flows through a pre pipeline, hits a provider adapter, then flows back through a post pipeline in reverse order:

          ┌──────────────────────── Gateway.complete(request) ────────────────────────┐
          │                                                                            │
 request  │   PRE  ─▶ PiiGuard ─▶ SecretScanner ─▶ PromptInjectionGuard ─▶ RateLimiter │
 ───────▶ │           CostControl ─▶ CostRouter ─▶ SemanticCache ──┐                   │
          │                                                        │ (cache miss)      │
          │                                             ┌──────────▼──────────┐        │
          │                                             │  Provider Adapter    │        │
          │                                             │ OpenAI · Anthropic · │        │
          │                                             │ Gemini · Azure ·     │        │
          │                                             │ OpenRouter · Ollama ·│        │
          │                                             │ Mock                 │        │
          │                                             └──────────┬──────────┘        │
          │           Guardrails ◀─ RiskScorer ◀─ PiiRestore ◀─────┘                   │
 ◀─────── │   POST ◀─ Metrics ◀─ AuditInterceptor (hash-chained record)                │ response
          │                                                                            │
          └────────────────────────────────────────────────────────────────────────────┘

• Interceptors implement before() / after() / onError(). Order is explicit — PII redaction runs before audit; audit runs last so it records what every other interceptor did. See docs/architecture.md.
• Executor policies (cache, retry, circuit breaker, load balancer, fallback) wrap the provider call itself — a cache hit or an open circuit short-circuits the provider entirely.
• The audit record is metadata-only. Prompts and responses are stored as SHA-256 hashes, never raw text; PII entity types and counts are logged, never values. Records are hash-chained (F-OBS-02) so any tampering is detectable.

Core data model — identical fields across all three SDKs, defined once in spec/ as JSON Schema and enforced by shared test vectors:

GavioRequest	GavioResponse	AuditRecord
trace_id (UUID v7)	trace_id	trace_id · parent_trace_id
agent_id · parent_trace_id	content (PII restored)	prompt_hash · response_hash
messages · model · provider	usage · cost_usd · latency_ms	pii_entity_types · risk_score
options · lineage · metadata	cache_hit · cache_type	previous_hash · lineage · schema_version

Runtime Intelligence

• AI Request Inspector — opt-in live traces, per-interceptor waterfall, replay, agent DAGs, RED stats, and production audit-store views.
• Runtime Event Export — metadata-safe JSONL and OpenTelemetry-style span streams for gateways, observability systems, and eval workflows.
• Ecosystem Integrations — compatibility matrix, metadata helpers, adapter payload helpers, conformance-generated trust evidence, and offline recipes for common gateways, observability tools, eval tools, frameworks, and provider SDKs.
• Prompt Registry + Evals — file-backed semver prompt manifests, approval metadata, metadata-safe diffs, prompt lineage, deterministic eval reports, gavio eval run CI gates, prompt-to-eval links, triage metadata, and prompt release bundles.
• Cost Intelligence — tenant/feature/user attribution, /api/cost-report, retry overhead, cache savings, and scoped budget fallback.
• Cost Governance v2 — budget policy/decision contracts, projected-spend controls, fallback/downgrade/dry-run actions, budget-aware reports, and gavio cost report.
• Domain-aware Policy Packs — signed catalog manifests for core, finance, healthcare, legal, HR, support, code security, and regional identifiers, plus custom regex-rule packs with overrides and false-positive suppression.
• Tool Runtime — validate tool inputs/outputs, freshness, conflicts, permissions, approvals, replay records, and MCP provenance before tool results re-enter model context.
• Runtime context — interceptors can now read first-class tenant, feature, cost, retry, tools, and policy fields derived from request metadata.
• Self-hosted Control Plane — runtime projects, environments, hashed keys, policy rollout, budget config, audit/event search, workflow release import, config snapshots, v2.3.0 durable SQLite/Postgres storage, v2.6.0 Enterprise Admin v2 controls, and v3.1.0 UX v2 for local/private deployments.
• Production Trust Package — build and verify metadata-only evidence bundles for release reviews, threat models, benchmarks, and production architecture signoff.
• Platform Runtime Profile — compute metadata-only readiness scores and deterministic gaps across audit, runtime events, policies, costs, tools, and trust evidence.
• Platform Workflow Release — produce reviewable release artifacts with prompt release bundles, eval gates, policy signatures, trust verification, and runtime-profile readiness.

---

Quick start (dev mode — no API key, no network)

import asyncio

from gavio import Gateway
from gavio.interceptors.pii import PiiGuard

async def main():
    gw = (Gateway.builder()
          .dev_mode(True)
          .use(PiiGuard())
          .build())

    r = await gw.complete(messages=[
      {"role": "user",
       "content": "mail [email protected]"}])
    print(r.content)        # PII restored
    print(r.audit.pii_entity_types)

asyncio.run(main())

import { Gateway } from 'gavio'
import { piiGuard } from 'gavio/interceptors/pii'

const gw = new Gateway({ devMode: true })
  .use(piiGuard())

const r = await gw.complete({ messages: [
  { role: 'user',
    content: 'mail [email protected]' }] })
console.log(r.content)   // PII restored
console.log(r.audit.piiEntityTypes)

import io.gavio.Gateway;
import io.gavio.GavioRequest;
import io.gavio.GavioResponse;
import io.gavio.interceptors.audit.AuditRecord;
import io.gavio.interceptors.pii.PiiGuard;

Gateway gw = Gateway.builder()
    .devMode(true)
    .use(new PiiGuard())
    .build();

GavioResponse r = gw.complete(GavioRequest.builder()
    .message("user", "mail [email protected]")
    .build()).join();

AuditRecord audit = (AuditRecord) r.audit();
System.out.println(r.content());
System.out.println(audit.piiEntityTypes());

All three print the reply with the email restored, and an audit record showing EMAIL was detected and redacted before the (mock) provider ever saw it. The Java snippet uses gavio-core, gavio-interceptor-pii, and gavio-interceptor-audit; the complete runnable project is examples/java/01-quickstart.

---

Install

Language	Command	Docs
Python 3.10+	pip install gavio==3.2.1	packages/gavio-py · docs/packages/python.md
JavaScript (Node 18+)	npm install [email protected]	packages/gavio-js · docs/packages/javascript.md
Java 17+ (Maven)	io.github.manojmallick:gavio-core:3.2.1 plus interceptor artifacts as needed	packages/gavio-java · docs/packages/java.md

---

Examples

Runnable examples live in examples/. Each row is the same scenario implemented per SDK wherever that surface exists, so the APIs can be compared side by side.

#	Scenario	Python	JavaScript	Java	Needs a key?
01	Quickstart — PII redact + restore, audit, cost	py	js	java	no
02	Production gateway — audit, PII guard, timeout, retry	py	js	java	optional
03	Custom scanner — write and test a PiiScanner	py	js	java	no
04	Production core stack — audit chain, PII, rate limit, guardrails, cache	py	—	—	no
05	Inspector & multi-agent tracing — waterfall, PII diff, agent DAG, sessions	py

Example 02 uses a real provider if ANTHROPIC_API_KEY or OPENAI_API_KEY is set; otherwise it falls back to the mock provider. All other examples run with no API key and no network.

---

Packages

Gavio is a monorepo. Each SDK is independently versioned-in-lockstep and published to its native registry.

🐍 Python — gavio (PyPI)

The reference implementation. Async-first (await gw.complete(...)), sync wrapper (complete_sync), full type hints + py.typed. Zero mandatory deps; optional extras include gavio[redis], gavio[presidio], gavio[otel], gavio[elasticsearch], gavio[pgvector], and gavio[ocr].

pip install gavio==3.2.1

→ Full Python guide · package README

🟨 JavaScript / TypeScript — gavio (npm)

Written in TypeScript, ships full type definitions, dual ESM + CJS build with per-subpath exports for tree-shaking. Native fetch, node:crypto. Node 18+, Deno, Bun.

npm install [email protected]

→ Full JavaScript guide · package README

☕ Java — io.github.manojmallick:gavio-* (Maven Central)

Multi-artifact Maven project: gavio-core plus one artifact per interceptor family (gavio-interceptor-pii, -audit, -reliability, -cache, -governance, -guardrails, -metrics, -quality), one per provider (gavio-provider-openai, -anthropic, -gemini, -azure, -openrouter, -ollama), and gavio-testing. Immutable records + builders, CompletableFuture async, Java 17+.

Quickstart stack:

<dependency>
  <groupId>io.github.manojmallick</groupId>
  <artifactId>gavio-core</artifactId>
  <version>3.2.1</version>
</dependency>
<dependency>
  <groupId>io.github.manojmallick</groupId>
  <artifactId>gavio-interceptor-pii</artifactId>
  <version>3.2.1</version>
</dependency>
<dependency>
  <groupId>io.github.manojmallick</groupId>
  <artifactId>gavio-interceptor-audit</artifactId>
  <version>3.2.1</version>
</dependency>

→ Full Java guide · package README

---

What ships

Every feature below lands in all three SDKs in lockstep, at the same version, gated by the same shared test vectors.

🔒 Privacy & security

Feature	ID	Since
PII Guard — Email, IBAN·mod-97, BSN·11-proef, CreditCard·Luhn, Phone, IP, SSN	F-SEC-01	0.1.0
Secret scanner — API keys, AWS AKIA, GitHub tokens, JWT, PEM, DB URLs	F-SEC-04	0.1.0
Prompt-injection defense — pattern corpus + optional semantic similarity	F-SEC-05	0.2.0
Embedding call guard — gw.embed(texts) runs the same PII pipeline before embedding APIs	F-SEC-10	0.9.0
Policy Pack architecture — core/FinTech manifests and custom regex-rule packs	F-PACK-01/02/05	0.12.0
Policy Pack Catalog — signed domain packs, load-by-name/path APIs, overrides, false-positive suppression	F-PACK-01/02/05	1.6.0

🔁 Reliability

Feature	ID	Since
Retry (exp backoff + jitter), Fallback chain, Timeout	F-REL-01/02/07	0.1.0
Circuit breaker, Load balancer (weighted round-robin)	F-REL-03/04	0.2.0
Streaming reliability — buffer response before post-interceptors run	F-REL-06	0.3.0
Loop guard — stop a runaway agent per root run: max steps, cost, wall time and repeated tool calls; stop, fall back or dry-run; verdict on the audit record. Python, JavaScript and Java	F-REL-09	3.2.1

💰 Cost & governance

Feature	ID	Since
Per-request cost_usd tracking (all providers)	F-GOV-01	0.1.0
Budget caps (soft/hard), rate limiting, model RBAC	F-GOV-02/03/04	0.2.0
Cost-optimiser routing — reroute simple prompts to a cheaper model	F-GOV-06	0.5.0
Cost Intelligence — tenant/feature/user attribution, cost reports, retry overhead, cache savings, scoped budget fallback	F-COST-01/02/04	0.11.0
Cost Governance v2 — policy/decision contracts, projected-spend controls, fallback/downgrade/dry-run actions, budget-aware reports	F-COST-05	1.2.0

⚡ Caching

Feature	ID	Since
Semantic + exact cache (cosine + SHA-256), in-memory backends	F-CACHE-01/02/03	0.2.0
Redis distributed backend (shared hits across processes, zero-dep RESP2)	F-CACHE-04	0.4.0

📊 Observability & quality

Feature	ID	Since
Audit interceptor + AuditRecord (SHA-256 hashes), stdout sink	F-OBS-01/05	0.1.0
Hash-chain (tamper-evident) audit, multi-agent DAG trace	F-OBS-02/03	0.2.0
Prompt lineage (template + variables + RAG sources)	F-OBS-04	0.3.0
Prometheus metrics (zero-dep text exposition)	F-OBS-08	0.3.0
OpenTelemetry-style span exporter and JSONL conversion	F-OBS-07	1.3.0
Prompt Registry + Evals — versioned templates, pass/fail cases, output hashes	F-EVAL-01/02	1.4.0
Eval Runner + CI Gates — gavio eval run, YAML/JSON suites, baseline comparison, JSON/JUnit reports	F-EVAL-03	2.1.0
Prompt Registry v2 — signed manifests, semver selectors, approval metadata, metadata-safe diffs	F-EVAL-04	2.2.0
Eval + Prompt Workflow — prompt/eval links, per-version gates, triage metadata, release bundles	F-EVAL-05	2.4.0
Guardrails — JSON-schema + regex allow/deny	F-QUA-01/02	0.2.0
Composite risk scoring (PII + guardrail + injection signals)	F-QUA-06	0.3.0
JSONL audit sink (jsonl://<path>) — the store the production dashboard reads	F-DX-08	0.7.0

🔬 Inspector

Feature	ID	Since
Dev-time visualizer — live traces (SSE), waterfalls, PII diffs, pipeline lints, embedded UI	F-DX-09/10	0.6.0
Agent call graphs + session views (/api/dag, /api/sessions)	F-OBS-10	0.7.0
Trace replay & edit-resend (full capture mode only)	F-DX-11	0.7.0
Read-only production dashboard — RED stats, hash-chain verifier, gavio inspect --store	F-DX-08	0.7.0
Export any trace as a PII-sanitized GavioTestKit test / test vector	F-DX-12	0.7.0
Overhead benchmarks with CI-enforced budget (<1% metadata / <5% full p50)	F-DX-09	0.8.0
Interceptor-chain overhead benchmark — fixed-seed Python/JS/Java harnesses and a validated benchmarks/latest.json (results, reproduce)	F-TRUST-02	3.2.0
Cost reports — /api/cost-report, top spend dimensions, retry overhead, cache savings	F-COST-02	0.11.0

🛠️ Developer experience & providers

Feature	ID	Since
Dev mode, dry-run mode, GavioTestKit	F-DX-01/02/03	0.1.0
OpenAI drop-in shim, config loader	F-DX-04/05	0.2.0
Runtime context fields — tenant, feature, cost, retry, tools, policy	F-RT-01	0.13.0
AI Request Runtime / Inspector positioning	F-DOC-V4	0.13.0
Tool Runtime — schema validation, freshness, conflict detection, provenance	F-TOOL-01/02/03/04	0.14.0
Tool Runtime v2 — registry-backed permissions, approval gates, replay, MCP metadata	F-TOOL-05/06/07/08	1.5.0
Normalised tool calling — one ToolCall / ToolResult model across OpenAI, Azure OpenAI, OpenRouter, Anthropic, Gemini, and Ollama; tool arguments pass through PII and secret scanning. Python in 3.2.0; JavaScript and Java in 3.2.1	F-ADP-03	3.2.0 / 3.2.1
Self-hosted Control Plane — runtime config, policy rollout, budget config, audit search, snapshots	—	1.7.0
Control Plane Persistence — SQLite migrations, Postgres adapter path, durable runtime/admin records	F-CP-01	2.3.0
Enterprise Admin v2 — OIDC/SAML-lite metadata, scoped admin keys, rollout approvals, audit export, retention controls	F-ADMIN-02	2.6.0
Control Plane UX v2 — admin UI, overview, demo seed, workflow release import, richer filters	F-CP-02	3.1.0
Production Trust Package — metadata-only release evidence bundle and verifier	F-TRUST-01	1.8.0
Ecosystem integration catalog — compatibility matrix, metadata helpers, JS subpath, and offline recipes	F-INT-01	1.9.0
Ecosystem adapter payloads — dependency-light payload fragments for LiteLLM, promptfoo, Langfuse, OpenLIT, LangChain, LangGraph, and Vercel AI SDK

Conformance-tested across all three SDKs on every push and PR (ci.yml runs Python 3.10–3.12, Node 18/20/22, Java 17/21). Per-release test totals are in the CHANGELOG; see the interceptors guide for every built-in interceptor.

---

Design principles

• P1 · Interface-first — every feature is a public interface you can swap or extend.
• P2 · Interceptor chain — pre/post hooks, explicit composition, no hidden magic.
• P3 · Provider-agnostic — no provider-specific code leaks into your app.
• P4 · Zero infra in dev — dev_mode runs everything in-process.
• P5 · Audit by default — opt-out, not opt-in.
• P6 · Embeddable library — runs in-process, no sidecar or proxy required.
• P7 · Dry-run first — log what would happen without blocking.
• P8 · Typed everywhere — TS generics, Python hints, Java generics.

---

Documentation

Doc	What
docs/getting-started.md	5-minute quickstart, all three languages
docs/cli.md	The gavio command-line tools with tested examples
docs/configuration.md	Provider keys, Inspector settings, cache directory, audit and event output
docs/troubleshooting.md	Fixes for common setup, Inspector and CLI problems
docs/architecture.md	Request lifecycle, the interceptor chain, data model
docs/interceptors.md	Every built-in interceptor + writing your own
docs/inspector.md	The Inspector: dev visualizer, agent DAGs, replay, production dashboard
docs/runtime-events.md	Runtime event/export contract and JSONL exporter
docs/prompt-registry-evals.md	Prompt Registry templates and metadata-safe eval reports
docs/control-plane.md	Self-hosted Control Plane UI, runtime config, policy rollout, budget config, workflow releases, and audit search

---

Repository layout

gavio/
├── spec/                     canonical data model (JSON Schema)
├── test-vectors/             shared cases every SDK must pass
├── packages/
│   ├── gavio-py/             Python SDK  (PyPI: gavio)
│   ├── gavio-js/             JS/TS SDK   (npm: gavio)
│   └── gavio-java/           Java SDK    (Maven: io.github.manojmallick:gavio-*)
├── examples/                 runnable cross-SDK examples
├── docs/                     documentation
└── .github/workflows/        ci.yml (test all 3) · release.yml (publish all 3)

---

License

MIT © 2026 Manoj Mallick

---

MIT © 2026 Manoj Mallick · Made in Amsterdam 🇳🇱