Open Knowledge Compiler
Compiles software engineering artifacts (Git history, PRs, Jira, source code, tests) into a structured, persistent knowledge base — queryable by humans (living wiki) and AI agents (MCP). Not another RAG system: raw artifacts go in once, compiled knowledge stays synchronized through incremental, PR-triggered compilation.
Inputs today: Git history, pull requests (GitHub), Jira issues linked from PRs, source code (Python, TypeScript, JavaScript) and the tests in it. Planned, not yet implemented: prose docs (README/Markdown) and OpenAPI specs — no collector or extractor exists for either yet.
Status: Architecture v1.0 frozen · V1 pipeline implemented (deterministic compiler for Python + TypeScript + JavaScript, incremental compilation with reconcile + verify, OKF v0.2-conformant wiki with loop-safe branch publishing and its own kc validate-okf checker, opt-in LLM semantic layer, opt-in embeddings with hybrid retrieval, read-only MCP server). Apache-2.0, dogfooded on real repositories including compiling itself (kc verify-clean, kc validate-okf-conformant). CI runs the full test suite on every push/PR against a real Postgres instance. Next: incremental --pr compilation wired into CI for this repo itself, and a real entry-point plugin ecosystem beyond the built-ins.
North star
Every design decision here should serve one question:
Given what this software does and what changed, what exactly must be verified — and how do we know the resulting test is actually trustworthy?
Structural knowledge (components, APIs, dependencies) is necessary but not the differentiator — generic code-intelligence tools already do that well. What doesn't exist elsewhere is the answer to the two halves of that question: a Behavioral Contract model (state, transitions, and failure modes — not just structure) for what must be verified, and a Test Trust Score (mutation-kill rate, flakiness, escaped-defect history, and coverage completeness combined into one signal) for is the resulting test trustworthy. That pairing is the moat.
Table of contents
Why OKF
The wiki this project compiles isn't a bespoke format — it's an OKF (Open Knowledge Format) bundle: plain Markdown with YAML frontmatter, readable by any editor and by any OKF-aware agent, with no proprietary SDK or vendor lock-in. OKF is deliberately minimal — the only universal requirement is a type field — and it explicitly separates producers (things that write knowledge) from consumers (things that read it), so tooling on either side can be swapped independently.
Hand-maintained OKF bundles (an Obsidian vault, a docs/ folder someone remembers to update) drift the moment nobody's looking. Open Knowledge Compiler is a producer: it compiles a conformant, always-current OKF bundle directly from your Git history, PRs, and code — no one has to remember to update it, because it's regenerated on every merge. See docs/okf-conformance.md for exactly how the emitted bundle maps to the spec, and kc validate-okf to check any bundle against it.
Prerequisites
- Python 3.11+ —
python --version to check
- Docker Desktop — for the Postgres database (download)
- Git — the repo you want to analyze must be a local git clone
NOTE - No API keys required for the base compile. LLM and embedding providers are opt-in (see below).
Installation
git clone https://github.com/kushal-omnius/open-knowledge-compiler.git
cd open-knowledge-compiler
# Create and activate a virtual environment
python -m venv venv
venv\Scripts\activate # Windows
# source venv/bin/activate # macOS / Linux
pip install -e .[all]
# Start Postgres (runs in Docker, port 5432, db=kc_wiki user=kc pass=kc)
docker compose up -d
Verify the install:
kc --help
See full CLI Documentation: kc-cli-reference.md
Analyze a repository
Point --dir at any local git clone.
Step 1 — Register the repo (run once):
kc init \
--slug my-service \
--forge-ref github.com/my-org/my-service \
--dir /path/to/my-service
This is the beginning of the README. Read the rest at the original. Read the full README ↗
