billion-context
Cache health at a glance: a healthy session keeps a 95–97% prefix-cache hit rate — compression itself costs ≤2%. Sustained lower? Check attribution with /acp or /acp-cache (see FAQ); usual causes, in order: upstream cache TTL expiry · model switch · a bili bug (please report) · other/unknown.
Community
QQ Group:
1056132097 (full)
1108730198 (full)
1129868472 (open)
📄 Paper / Preprint
📝 The paper itself is open-sourced under the MIT License as part of the codebase (paper/). It is a living document — anyone may edit it; improvements are welcome via pull request.
A production-scale longitudinal study: 4.5 months, three hosts, 174,327 model calls, 18.76B cumulative input tokens (~24.7B across all hosts), zero window violations on 204,800-token models, marathon sessions of 8,584–12,049 calls.
📊 Independent Third-Party Benchmarks
- 1,500-Item Cross-Benchmark Evaluation (
LongMemEval-S · ContextBench · MemoryAgentBench) (中文版) — head-to-head comparison of billion-context (v0.1.202) against full context and 3 RAG arms (bm25, embedding, embedding-rerank) with DeepSeek V4.1 Flash, Qwen3-Embedding-8B, Qwen3-Reranker-4B, and Pi Agent: #1 on ContextBench (0.3097 Utilized F1, -58.3% cost vs full), +16.6% over best RAG on LongMemEval-S (79.20% vs 62.60%, -71.1% independent-session cost vs full), and 100% survival on 1.48M-token MemoryAgentBench TTL (30× steady-state compression where full fails 199/199).
billion-context sits between any agent and its model API, rewriting Anthropic/OpenAI streams with acp-kernel compression. The model decides when and what to compress into high-fidelity summaries — not a hard truncation limit.
Why
Long coding sessions blow up context. Each provider charges per token, and once you pass the context window the session degrades or dies. billion-context compresses consumed conversation into layered summaries so you can run a single session for days — billions of tokens through one context window.
Unlike a host's built-in summarizer, compression here is incremental, reversible, and prefix-cache friendly: summaries are written in small ranges, can be decompressed on demand, and the cache prefix stays intact.
How it works
Context-management tools
The proxy injects four context-management tools into the conversation; the model calls them itself as context grows, and the proxy executes compress server-side so folded ranges stay summarized in history until restored:
compress — fold a message range into a detailed summary.decompress — restore a compressed range when exact details are needed again.search_context — keyword search over compressed summaries and visible messages.acp_status — context-usage overview plus which ranges are still compressible.
Which do I need?
Pick by your client:
This is the beginning of the README. Read the rest at the original. Read the full README ↗
