Niyam
When the law changes, the code changes with it.
Niyam (Hindi and Marathi for "rule") is an AI agent that reads new government orders and keeps welfare eligibility rules, written as code, in sync with the law. Every rule links to the clause it came from, every change is tested and independently verified, and the agent's full reasoning is saved with the change.
Built during neatHack 2026 by team USV.
Demo video: youtu.be/LI24QLu4gzc. neatHack submission: quote post with the 2-minute demo.
The problem, in one paragraph
In India, eligibility for welfare is decided by software: portals, field-worker apps, helplines, chatbots. The law behind it is published as PDF Government Resolutions (GRs), mostly in Marathi in Maharashtra, and it changes often. Nobody keeps the code in sync. When code drifts from the law, people are wrongly excluded or wrongly paid: in Telangana an eligibility algorithm cancelled more than 1.8 million food cards (Al Jazeera and Pulitzer Center, 2024), and nobody affected could see why.
What Niyam does
Watch -> Triage -> Extract -> Encode -> Verify -> Resolve -> Publish
- Watch the public list of new Maharashtra GRs every 15 minutes (never the CAPTCHA-protected search, never bypassing anything).
- Triage with a fast model: does this order change a covered scheme?
- Extract every eligibility change with the exact Marathi quote, page and clause. Each quote is checked against the PDF; invented quotes and wrong numbers are rejected.
- Encode: Claude Code edits the rulebook in its own git worktree, adds dated values (history is never overwritten), and writes test citizens at every limit. It never sees the raw PDF, only the checked change list.
- Verify: a separate agent on a different model, blind to the code change, writes its own test citizens from the law. They are run against the new rules.
- Resolve: a coordinator settles disagreements, sends precise fixes, or asks a human a plain-English question.
- Publish: a pull request with quotes, verification, confidence and reasons. Entire links the coding agent's session to the commit as a checkpoint. It merges on its own only when the trust policy allows: numbers or dates only, every check agrees, confidence at least 0.90, and nobody loses eligibility.
Results (measured, small sample)
Six evaluation cases: the real 3 July 2024 Ladki Bahin amendment, the same order as a scan with no text layer, two real unrelated GRs, and two synthetic robustness tests labelled as synthetic inside the PDF (a pure benefit increase, and a hidden prompt injection).
| Metric | v1 | v2 | v3 |
|---|
| Changes found, values, boundaries | 100% | 100% | 100% |
| Test citizens get the right answer | 97.6% | 100% | 100% |
| Correct choice to act alone or ask a human | 67% | 100% | 100% |
| Independent verifier agrees | 95.7% | 100% | 100% |
| Cost per document | $0.88 | $0.29 | $0.13 |
| Time per document | 110 s | 54 s | 40 s |
v3 is a measured model choice. Per-stage cost tracking showed the coding agent was 56% of the cost. We tested cheaper models stage by stage on the same cases: Haiku 5.5 for triage and for the coding agent, and Sonnet 5.5 instead of Opus 5.5 for the verifier, kept 100% quality. Haiku 5.5 for extraction did not: it reported "no changes" for a real age-limit change, so extraction stays on Sonnet 5.5. The experiment also exposed a bug (a cut-off structured answer crashed instead of recovering), now fixed and unit tested. All runs: evals/results/extra/v3a.json, v3b.json, v3c.json.
Further runs (in evals/results/extra/):
- A whole scheme from its founding order. From an empty rulebook and only the real 28 June 2024 GR, Niyam recognised a new scheme, extracted 19 rules, wrote the rulebook and 34 tests, and passed 25 of 25 of our hand-written baseline test citizens, which it never saw ($0.81, 201 s, checkpoint
01M4JFNT0M069MG48K0FT9V50P).
- Consistency. 9 more v2 runs on the three relevant cases: 100% on every metric, $0.61 per relevant order.
- Recovery under injected failures. A broken model output and an unusable text layer injected on purpose: both recovered, 100% outcomes.
- A second real scheme. On an operator's request (
--onboard), Niyam encoded Sanjay Gandhi Niradhar Anudan Yojana from the real 56-page 2019 consolidated GR (PR #3, 49 tests, $1.54). Then the real 6 July 2023 order raising assistance to Rs 1,500 was judged safe to merge alone (only amounts rise, 16 of 16 checks agree, confidence 0.98, $0.20, dry run). Run live, the same order raised a question about widows with three or more children and went to a human (PR #4): 2 of 4 runs would have acted alone; the other 2 asked a person, the safe direction. Reviewing PR #3 caught a real mistake: a new disabled field defaulted to true, which would widen eligibility; fixed before merge. All four agent PRs are now reviewed: #2, #3 and #4 merged, #1 closed as superseded.
- Acting alone. A labelled synthetic order that only adds people: 2 of 2 runs decided to merge alone (dry run, never merged into the public rulebook). Every run on the real amendment, which removes some people, asked a human.
The improvement came from the neatlogs traces: in v1 the coordinator ruled three different ways on one test case and the job spent $3.05 before the budget stopped it. The root cause was that the benefit formula ("pay the difference") was missing from the context given to the verifier and coordinator. v2 adds it, plus a loop guard. One v2 run was discarded because of a bug in our own file guard; the full, honest story is in hackathon.md. Raw results: evals/results.
Evidence
| Claim | Evidence |
|---|
| The agent completes a substantial task end to end | PR #2: a Marathi GR in, 6 changes encoded with 46 passing tests, 16 of 16 independent checks agree |
| The agent improved | evals/results/v1.json and v2.json, with neatlogs trace ids for every run |
| We used neatlogs | One trace per job, a span per stage and per recovery; trace ids in every eval result and PR |
| We used Entire | Checkpoints on every agent commit in the rulebook (for example 01M4JDH944E4HPRCA4WKDJTFN7) and on build commits in this repository |
| We used cfo.ai | 36-month plan built by cfo.ai's Ari, three cases; PDF; inputs and summary in business/ |
| The agent recovers from failures | Scan with no text layer recovered by transcription (recovery.ocr_fallback); a crashed job resumed from its last good state; loop guard escalation |
| Code written during the event | Both repositories created after 12:30 IST on 10 October 2026 |
How the sponsor tools are used
- neatlogs: traces every model call, tool call, retry and cost. We used the traces to find the v1 loop, fix it, and verify the fix with before and after numbers.
- Entire: legal provenance. Every rule change by the agent is a commit with an Entire checkpoint holding the coding agent's full session, so anyone can ask "why does this rule say this?". Uses
commit_linking: always so the headless agent's sessions link without a prompt.
- cfo.ai: Ari built the 36-month business plan (downside, base, upside) from our prices and the cost per document measured in neatlogs. Base case: INR 2.51 crore revenue in year 3, about 95% gross margin, cash flow positive from month 21, INR 40.5 lakh funding need beyond the starting INR 25 lakh.
- Claude: chosen per stage by measurement: Sonnet 5.5 extracts, verifies and resolves; Haiku 5.5 triages and runs Claude Code as the coding agent.
Use it
# API (eligibility with sources, rules on any date, jobs with live events, BYOD, keys)
cd agent && uv venv && uv pip install -e . && uvicorn niyam.api:app --port 8000
# Run the agent on a government order
python -m niyam process path/to/gr.pdf --date 2024-07-03
# Evaluations
python ../evals/run_evals.py --version v3
# MCP server for AI assistants
claude mcp add niyam --env NIYAM_API_BASE=http://localhost:8000 -- uvx --from "git+https://github.com/usv240/niyam#subdirectory=agent" niyam-mcp
# Website (static export; works without the API)
cd web && npm install && npx next build
Keys go in a .env file outside the repository (see .env.example). Needs ANTHROPIC_API_KEY, and NEATLOGS_API_KEY for traces.
Repository layout
| Path | What |
|---|
agent/niyam/ | The agent: orchestrator.py (state machine), stages/ (triage, extract, encode, verify, resolve, publish), pdftext.py (citation checker), confidence.py (trust policy), api.py, mcp_server.py, sources/watcher.py |
agent/tests/ | Tests for the citation checker, trust policy, watcher and API privacy |
evals/ | Harness, cases (real and labelled synthetic), results per version |
web/ | Website: landing, demo replay, rule explorer, eligibility checker (English, Hindi, Marathi), BYOD, developers, judges, learn, trust, status |
scripts/export_site_data.py | Builds the website's data from the rulebook and real runs |
business/ | Business plan |
infra/ | Deployment |
Limitations
- One scheme so far (Mukhyamantri Majhi Ladki Bahin Yojana), from its real orders.
- Small evaluation set. Only eligibility and amounts are modelled, not documents or procedure.
- Hindi and Marathi interface text needs review by a native speaker.
- Not an official government service. Information only; the government department decides.
License
MIT
