Scopewatch

Check every agent action before it runs. A pre-execution gateway for AI agents, built for the Nebius × NVIDIA Global AI Hackathon 2026.

Website · Walkthrough video, 2:42 · Live demo replay · Project brief (PDF) · Judge runbook · Project status

Coding agents run shell commands and edit files, and a prompt injection in a README or invoice can steer them. Scopewatch checks each tool action before it runs. A deterministic policy returns ALLOW, HOLD, or DENY. An auditor model reads the agent's exposed reasoning and can hold an allowed action for human review. Every step is recorded as evidence in a reviewer dashboard. It mediates only actions routed through its API.

Run it (no keys, no Docker, no network):

python3 -m venv .venv && . .venv/bin/activate && pip install --require-hashes -r backend/requirements.lock && ./scripts/run_demo.sh

Then open http://127.0.0.1:8000/. Needs Python 3.12+. Node 22+ is only for the frontend test suites (BUILDING.md has details).

Also: hackathon checklist · submission package · website source in site/

How it works

1. Policy checks paths, tools, and operations first. Network operations are currently denied, even when the task scope lists destinations. A DENY is final.
2. The auditor can escalate a policy ALLOW to HOLD. An audit that was attempted and failed also holds the action. The model cannot grant access.
3. A held action needs an exact, single-use approval before the executor runs it. Approval cannot override a policy DENY.
4. The dashboard separates observed actions, policy decisions, provider-exposed traces, summaries, approvals, and execution receipts. A capture credential gates verified provenance labels. Grounded excerpts quote the submitted evidence and do not prove intent.

Reviewer dashboard screenshots, with how each was captured, are in the media gallery.

Adoption and prior art covers who this is for, real vulnerability disclosures, overlap with existing guardrails, and limits under monitoring awareness. No adoption-time or cost savings have been measured. The shipped agent/auditor pair and the live headline scenario are still open (#214).

What the evidence does and does not show

• Interception coverage. Scopewatch is an application-level gateway. Actions that bypass the API are not observed, blocked, or recorded, and it does not intercept host processes or direct system calls. A Docker-isolated executor runs run_command (backend/scopewatch/executor_docker.py).
• Reasoning. The auditor receives raw reasoning only from provider response fields, checked in order in backend/scopewatch/providers/client.py: message.reasoning_content, message.reasoning, then message.reasoning_details. In reasoning_details, reasoning.text blocks are provider-exposed traces (PROVIDER_EXPOSED_TRACE), reasoning.summary blocks are summaries (AGENT_AUTHORED_SUMMARY), and redacted or unknown variants are UNAVAILABLE. Anything else, including <thinking> text inside message content, is ignored and recorded as UNAVAILABLE. Traces go to the auditor inside <untrusted_reasoning_trace> delimiters. A model can rationalize, omit, or misstate its plan, so a clean trace never proves benign intent. Closed models usually expose no reasoning fields.
• Provenance. The API cannot tell a trace captured by the provider client from a label typed into a JSON body. Only a submission carrying the operator-issued SCOPEWATCH_CAPTURE_TOKEN is stored as a verified provider trace. Other claims are stored as CALLER_ASSERTED_PROVIDER_TRACE or CALLER_ASSERTED_SUMMARY. Labels are evidence metadata and never change a policy decision.
• Model data policy. Only synthetic fixtures and invented content go to model providers. Free and stealth endpoints may log prompts. Evaluation reports pin the profile, served model ID, date, and case-file hash (evaluation.md).

---

Other ways to run

./scripts/run_demo.sh seeds the synthetic scenarios, serves the dashboard on 127.0.0.1:8000, and replays them with no model. Override with PORT=... HOST=.... State goes to runtime-data/scopewatch.db (gitignored) and per-run copies of demo/workspace/.

./scripts/run_demo.sh --agent                  # agent loop with the offline mock provider
./scripts/run_demo.sh --coding                 # coding scenarios 10-13 (needs Docker, see below)
./scripts/run_demo.sh --db-path /tmp/sw/demo.db --workspace-root /tmp/sw/workspace   # keep the repo clean

Scenarios 10 and 13 submit run_command, which policy permits only on the Docker executor. With a reachable Docker daemon the launcher selects SCOPEWATCH_EXECUTOR=docker. Without one it warns before seeding, and every run_command is refused by policy.

Live Nebius inference (needs NEBIUS_API_KEY; keys come from the environment or a gitignored .env, see .env.example):

export NEBIUS_API_KEY="..."
export SCOPEWATCH_AGENT_PROFILE=nebius-demo SCOPEWATCH_AUDITOR_PROFILE=nebius-demo
./scripts/run_demo.sh --agent --profile nebius-demo

여기까지가 README 앞부분입니다. 나머지는 원문에서 읽을 수 있습니다. README 전문 보기 ↗