HALO

Harness for Always-On Loop Oversight

Overview · MCP plugin · Setup · Read mode · Run mode · Results · Cases · Citation

Overview

HALO is a diagnostic tool that assesses the loop completeness of existing always-on assistants. It follows the framework of the survey Always-On Personal Intelligence: an always-on assistant runs a continuing loop of three functional modules, Sensing, Memory, and Action, connected by six directed coupling channels, and self-evolution extends this loop when the assistant revises how it works from its own operating experience.

HALO examines two parts of this framework: the coupling channels of the continuing loop and, for self-evolution that rests on outcome feedback, the five steps of the chain of evidence. For each channel or step, it asks two questions and answers them in two modes.

The five steps are record, outcome, attribution, revision, and later use. For an email assistant that learns from the user's edits, as EAIA does:

Record	Outcome	Attribution	Revision	Later use
the draft it wrote	the user's edit	links the edit to that draft	rewrites the drafting prompt	the next draft, written with the new prompt

[!NOTE] The word harness in the name refers to these two modes of analysis, not to the harness inside an assistant's scaffolding.

MCP plugin

HALO also works as an MCP plugin for agents that are already running, such as ChatGPT dots, Grok Bot, Claude Code, or Codex. The agent logs each step of its loop to HALO, naming the earlier steps it used, and HALO reads the channels, the evidence steps, and the loop-closure level the trace supports from those links:

uv pip install -p .venv/bin/python -e ".[mcp]"
halo serve                      # stdio, for local hosts
halo serve --http --port 8765   # Streamable HTTP
halo trace report <agent>       # the diagnosis of one agent's trace

Any MCP client can start it with:

{
  "mcpServers": {
    "halo": { "command": "halo", "args": ["serve"] }
  }
}

The documentation, built from site/ (an Astro Starlight site), explains the plugin's architecture, how to connect each host, and the MCP tools.

What is in this repository

halo/                 the package
  mcp/                the MCP plugin: server, ledger, analysis, and `halo trace`
  backends/           model calls through local coding-agent CLIs (Codex, Claude Code)
  read/               read mode: protocol, review, batch, vote, tables
  run/                run mode: adapter base class, worker, harness, judge, cached LLM
cases/<work>/         one folder per reviewed work (40 works)
  case.yaml           the configuration reviewed, its sources, and its code
  core/               the upstream code that run mode executes, copied unchanged
  adapter.py          connects the upstream code to the harness and defines the cut sites
  run.yaml, inputs/   the run-mode tests and their scripted inputs
results/read/         our read-mode reviews, votes, per-work counts, and the table
results/run/          our run-mode reports
docs/                 how to add a case or connect your own system
site/                 the documentation site for the plugin
scripts/              cloning the upstream repositories, generating case files

Setup

uv venv .venv -p 3.11
uv pip install -p .venv/bin/python -e .

Models are called through local CLIs, so no API key is stored in the repository. Point HALO at them with environment variables:

# backend names codex:<model>, e.g. codex:gpt-6-astra
export HALO_CODEX_BIN=/path/to/codex
# backend names claude:<model>, e.g. claude:claude-opus-5-5
export HALO_CLAUDE_BIN=/path/to/claude

Each call runs without the local account's own configuration, memories, or rules (halo/backends/cli.py).

Read mode

python scripts/clone_upstream.py                         # code the agents read, at pinned commits
halo read --backend codex:gpt-6-astra --workers 6        # one oversight agent over all works
halo read --backend claude:claude-opus-5-5 --workers 6
halo read --backend claude:claude-fable-5-1 --workers 6

halo vote      # results/read/votes.json, disagreements.md
halo tables    # results/read/table.md, table.csv, counts.json

The protocol every agent follows is halo/read/protocol/PROTOCOL.md, and every review must validate against halo/read/schema.json.

Run mode

uv venv cases/aiget/.venv -p 3.11
uv pip install -p cases/aiget/.venv/bin/python -r cases/aiget/requirements.txt
halo run cases/aiget --workers 4

The harness writes runs/<case>/<name>/ with every arm's output (arms.jsonl), the judge's verdicts (judge.jsonl), and a summary (report.md, report.json). Model replies are cached, so arms whose prompts are identical share them and a rerun costs nothing.

Our results

Read mode

Three oversight agents reviewed all 41 configurations of the 40 works: GPT-6-Astra (Codex CLI, reasoning effort ultra), Claude Opus 5.5 and Claude Fable 5.1 (Claude Code CLI, effort max). Counts over 40 works, from the majority vote (results/read/counts.json; a work with several configurations counts once):

	S→M	M→A	A→S	M→S	A→M
reported	38	40	30	25	31
insufficient	0	0	8	13	7
absent	2	0	2	2	2

Evidence steps stated:

record	outcome	attribution	revision	later use
30	20	26	6	3

19 works report A→M and also retain outcomes.

Every review, with quotes and locators, is in results/read/reviews/<agent>/; the vote is in results/read/votes.json, the cells without a unanimous vote in results/read/disagreements.md, and the table in results/read/table.md.

This is the beginning of the README. Read the rest at the original. Read the full README ↗