autoloop

Config-driven AI pipeline that triages GitHub issues, implements them via Claude, and opens PRs. One config file. One command to start.

Prerequisites

• Python 3.13+
• uv (recommended): install or use pip instead
• Node.js 22+: required by Claude Code CLI
• GitHub CLI (gh): install then run gh auth login
• Claude Code CLI (claude) v2.1.310 or later: npm install -g @anthropic-ai/claude-code@latest, then check with claude --version (requires a Claude Max or Pro subscription)
  • Which Claude models you can use depends on the installed claude CLI version (and your plan). autoloop passes triage_model, impl_model and review_model straight to claude --model, so a model newer than your CLI fails at run time. Upgrade the CLI before switching to a newer model.
  • The default impl_model (and review_model) is claude-opus-5-5, which needs CLI v2.1.310 or later. Older CLIs such as 2.1.160 can't run it. Upgrade, or set impl_model in autoloop.toml to a model your CLI supports.
  • Implement runs pass permission rules with --settings, so the CLI must support that flag. autoloop doctor checks this.
• A GitHub repo with a test command (pytest, npm test, make test, etc.)

TLDR: First PR in 5 Minutes

uv tool install git+https://github.com/Sanctum-Origo-Systems/[email protected]
cd your-repo
autoloop init --repo your-org/your-repo --verify-cmd "npm test"
autoloop doctor                           # verify environment
# Create a GitHub issue with a clear title and acceptance criteria
autoloop triage --drain                   # evaluates and labels issues
autoloop implement --auto-fix             # builds, reviews, fixes if needed
# Review and merge the PR

Session conflict detection: All commands that launch claude -p (implement, triage, review-pr, fix-pr, plan) check for active Claude Code sessions and exit with a clear message if one is detected. To avoid conflicts, launch your interactive Claude Code session from the parent folder, or enable worktree isolation to run autoloop in a separate working tree.

That's it. Read on for common workflows, configuration, scheduling, and issue writing tips.

---

Common Workflows

# Triage
autoloop triage                            # single pass, all untriaged issues
autoloop triage --drain                    # loop until all sub-issues triaged
autoloop triage --issue 42                 # triage a specific issue

# Implement with automated review
autoloop implement --auto-fix              # implement + review + fix loop
autoloop implement --max-issues 5          # batch mode
autoloop implement --auto-fix --max-issues 5  # batch with auto-fix

# PR management
autoloop review-pr 42                      # review an existing PR
autoloop fix-pr 42                         # fix a broken PR

# Diagnostics
autoloop status                            # check pipeline state
autoloop doctor                            # verify environment

Pipeline

Issue --> Triage --> Implement --> Review PR
                                      |
                                 Pass? --> PR ready for human merge
                                 Fail? --> Fix PR --> Review again (up to 3 rounds)
                                              |
                                 Exhausted? --> needs-human label

View live performance dashboard (EVAL.md) — first-attempt success rate, cost trends, and per-module breakdown, with a Definitions section (numerator, denominator, run types and window for each metric), pending counts for open PRs, and a "data as of" stamp that flags history more than 48 h old. Updated daily.

View the decision-model shadow report (JEV.md) — how often Jev agrees with triage and auto-merge decisions, and whether it's ready to act. Updated daily.

---

Platform Support

• macOS: fully supported (local mode)
• Linux: fully supported (local + unattended/VPS mode)
• Windows: not supported

How It Works

	Local mode	Unattended mode
Where	Your machine	VPS / remote server
Trigger	You run the command	systemd timer / cron
Gate	You review + merge PRs	Same. You're still the human gate
Best for	Evaluating autoloop, small projects	Trusted repos, overnight batch runs

Most users start local. Graduate to unattended when you trust it.

Installation

As a CLI tool (recommended)

Replace <tag> with the latest version from the tags page (e.g. v0.19.0). Pinning a tag is recommended; @main tracks unreleased changes:

uv tool install git+https://github.com/Sanctum-Origo-Systems/autoloop@<tag>

Verify it worked:

autoloop version

Alternative: pip

pip install git+https://github.com/Sanctum-Origo-Systems/autoloop@<tag>

Upgrading

# uv for a specific version
uv tool install --force git+https://github.com/Sanctum-Origo-Systems/autoloop@v<new-version>

# uv for the latest version
uv tool install --force git+https://github.com/Sanctum-Origo-Systems/autoloop@main

# pip for a specific version
pip install --upgrade git+https://github.com/Sanctum-Origo-Systems/autoloop@v<new-version>

# pip for the latest version
pip install --upgrade git+https://github.com/Sanctum-Origo-Systems/autoloop@main

Verify: autoloop version

If you use the MCP server, include the mcp extra when you upgrade: uv tool install --force "autoloop[mcp] @ git+https://github.com/Sanctum-Origo-Systems/autoloop@v<new-version>". --force rebuilds the tool environment with only the extras you name, so leaving it out removes fastmcp and autoloop-mcp stops starting.

If running on a VPS with systemd timers, reinstall there too and restart any Claude Code sessions in tmux to pick up the new MCP tools.

For contributors

git clone https://github.com/Sanctum-Origo-Systems/autoloop.git
cd autoloop
uv sync
uv run autoloop version

Contributions are currently limited to collaborators. A public issue board for ideas and feature requests will be available soon. In the meantime, reach out to [email protected].

Quick Start (Local Mode)

1. Initialize your repo

cd ~/my-project
autoloop init --repo owner/repo --reviewer your-github-username --verify-cmd "pytest"

This creates three files:

File	Purpose
autoloop.toml	Pipeline configuration (models, timeouts, verify command)
.github/workflows/autoloop-cleanup.yml	CI workflow that cleans labels and auto-closes parent issues on merge
.gitignore	Adds autoloop/run_history.jsonl and .autoloop.lock

It also creates GitHub labels (ready, rejected, in-progress, etc.) used by the triage system.

Operational data (run_history.jsonl, jev_decisions.jsonl) is stored outside the repo tree in ~/.autoloop/<id>/, keyed by git remote URL. This keeps operational data out of your working tree and supports multiple repos on the same machine. Daily eval snapshots are stored in ~/.autoloop/<id>/eval_snapshots/. Only published snapshots are committed to autoloop/eval_snapshots/ together with EVAL.md (via PRs with --publish --pr). eval --publish compares against the last published snapshot and publishes only on a material change, so a no-change run leaves the working tree clean. Add --force to publish anyway, e.g. to roll out a fix to how metrics are calculated or formatted. If chore/eval-<date> already has an open PR, --pr adds the new commit to that PR. A stale local branch (merged, or with a merged or closed PR) is replaced. Any other existing branch gets a suffix (-2, -3, …). If publishing fails, EVAL.md and the snapshot are restored on main.

Commit the generated files:

git add autoloop.toml .github/workflows/autoloop-cleanup.yml .gitignore
git commit -m "feat: add autoloop pipeline"
git push

Local Mode Notes

• autoloop init scaffolds the required .claude/settings.json permissions. If you skipped init, create one manually (see template).
• Session detection guards all commands that launch claude -p. If a conflict is detected, autoloop exits with a message. Launch your Claude Code session from the parent folder, or enable worktree isolation.

2. Verify your environment

autoloop doctor

Checks that everything is set up correctly before your first run:

✓ autoloop.toml found and valid
✓ .claude/settings.json found
✓ claude CLI installed (v2.1.310)
✓ claude CLI authenticated
✓ gh CLI installed and authenticated
✓ No active Claude Code session conflict
✓ verify_cmd passes ("pytest" → exit 0)
✓ .autoloop.lock is ignored and untracked
✓ origin/main:.claude/settings.json valid (12 allow rules)
✓ claude CLI supports --settings

Implement runs pass origin/main:.claude/settings.json to claude -p via --settings. Doctor fails if that file can't be read or the installed CLI lacks --settings, because implement runs would then have no permission rules. It warns if permissions.allow is empty.

autoloop doctor --live also runs a paid probe (off by default): it calls claude -p in a throwaway worktree of origin/main and passes if the agent can write and commit a file and its git push to a nonexistent remote is denied. The worktree and branch are removed afterwards, and the probe's cost is printed.

Each check reports pass or fail with a fix suggestion. Exit code 0 if all pass, 1 if any fail. Run this once after autoloop init and again if implementations start failing unexpectedly.

3. Configure

Review autoloop.toml and adjust for your project. Key fields:

repo = "owner/repo"              # Your GitHub repo
verify_cmd = "pytest"             # Command that validates the implementation
lint_command = "ruff check && ruff format --check"
impl_model = "claude-opus-5-5"  # Model for implementation
triage_model = "sonnet"           # Model for triage (lighter, cheaper)
max_retries = 3                   # Retry attempts per issue
protected_paths = ["autoloop.toml"]  # Files the bot must never modify

4. Create an issue

Write a GitHub issue with clear structure. Autoloop works best with specific, testable issues.

Good issue:

## Summary
Add a --verbose flag to the CLI that prints each processed file

## Type
feature

## Expected Behavior
When --verbose is passed, each file name is printed to stdout as it's processed.
Without --verbose, only the summary count is shown.

## Acceptance Criteria
- [ ] --verbose flag accepted by argument parser
- [ ] Each file printed on its own line when flag is set
- [ ] Default behavior unchanged without the flag

Bad issue (will be rejected by triage):

Make the CLI better. It should show more stuff.

You don't need to list which files to modify. Triage identifies the relevant files automatically from the issue description and codebase. Including file paths is helpful if you know them, but not required.

Issues can be any size. If an issue is too large, triage automatically decomposes it into ordered sub-issues with dependency tracking.

To order issues yourself, add a Depends on: line to the issue body, e.g. Depends on: #12, #15. Implement skips the issue until every listed issue is closed. If a dependency's state can't be read, the issue counts as blocked.

5. Triage

autoloop triage                         # triage all untriaged issues (single pass)
autoloop triage --drain                 # loop until all sub-issues are triaged
autoloop triage --drain --max-rounds 3  # custom safety limit (default: 5)
autoloop triage --issue 42              # triage a specific issue

By default, triage runs a single pass over untriaged issues. Use --drain to loop until no untriaged issues remain — this is useful when decomposition creates new sub-issues that themselves need triage. The --max-rounds flag sets a safety limit on drain iterations (default 5).

Triage evaluates each untriaged issue and applies a label:

Label	Meaning
ready	Template complete, small enough to implement
needs-decomposition	Template complete but too large. Sub-issues are created automatically
rejected	Missing required fields (autoloop attempts to auto-fix and re-triage once)
needs-human	Touches protected paths or requires human judgment

6. Implement

autoloop implement              # implements the top ready issue
autoloop implement --issue 42   # implements a specific issue
autoloop implement --max-issues 5  # implements up to 5 issues in sequence
autoloop implement --auto-fix   # auto-fix PR review findings (up to max_pr_review_rounds)
autoloop implement --auto-fix --max-pr-review-rounds 5  # override max rounds

For each issue, autoloop:

1. Creates a feature branch
2. Runs Claude to implement the changes
3. Runs your verify_cmd and lint checks
4. Reviews the implementation for quality
5. Retries on failure (up to max_retries)
6. Opens a PR with run stats (duration, cost, tokens)

7. Review and merge

Autoloop opens the PR but never merges it. You review and merge. That's the human gate. On merge, the CI workflow cleans up labels and auto-closes parent issues when all sub-issues are complete.

Merge queue (optional)

The CI workflow triggers on merge_group events, so it works with GitHub's merge queue. To enable it:

1. Go to Settings > Rules > Rulesets (or Settings > Branches for classic branch protection)
2. Enable Require merge queue on your target branch
3. Under Required status checks, add the test job from the CI workflow
4. PRs will be queued, rebased, and CI-verified before merging

8. Fix broken PRs

autoloop fix-pr 42

Detects and fixes:

• Stale base: rebases on main
• Merge conflicts: rebases, Claude resolves conflicts
• Lint failures: runs ruff fix/format, falls back to Claude
• Test failures: Claude fixes the code, re-verifies

9. Review a PR

autoloop review-pr 42

Runs the review stage on an existing PR without merging:

1. Checks out the PR branch
2. Runs the mutation gate (tests, lint, test-file check)
3. Runs a semantic review via Claude using the review_model
4. Posts findings as a PR comment
5. Applies the needs-human label if either gate or review fails

Use this to get automated review feedback on PRs that were opened manually or by external contributors. The command never merges — you always make the final call.

Creating Issues from a Spec

autoloop plan --from-spec path/to/spec.md

Parses a markdown spec or PRD file for ## Task N: sections and creates a GitHub issue for each one. The legacy ## Enhancement tag is also supported for backward compatibility.

Example spec format:

## Task 1: Add user authentication

Add login and logout endpoints with session management.

**Problem:** No auth on API endpoints.

**File:** `src/api/auth.py`

## Task 2: Add rate limiting

**Problem:** API allows unlimited requests.

**File:** `src/api/middleware.py`

The **File:** section is optional. Include it if you know which files need modification. The builder will determine the correct files from the description if omitted.

Each section becomes one issue. Triage then handles decomposition and dependency ordering if any issue is too large.

Running Unattended

Once you trust the pipeline, automate it with timers on a Linux VPS.

With systemd timers (Linux VPS)

Create a service and timer for both triage and implement. Adjust OnCalendar to set the frequency:

# ~/.config/systemd/user/myapp-triage.service
[Service]
Type=oneshot
WorkingDirectory=/home/user/my-project
ExecStart=/home/user/.local/bin/autoloop triage

# ~/.config/systemd/user/myapp-triage.timer
[Timer]
OnCalendar=*-*-* 00:00:00 UTC    # once daily at midnight
Persistent=true

[Install]
WantedBy=timers.target

# ~/.config/systemd/user/myapp-implement.service
[Service]
Type=oneshot
WorkingDirectory=/home/user/my-project
ExecStart=/home/user/.local/bin/autoloop implement --max-issues 5
TimeoutStartSec=7200

# ~/.config/systemd/user/myapp-implement.timer
[Timer]
OnCalendar=*-*-* 02:00:00 UTC    # once daily at 2am
Persistent=true

[Install]
WantedBy=timers.target

Common frequencies:

• Daily: OnCalendar=*-*-* 00:00:00 UTC
• Every 6 hours: OnCalendar=*-*-* 00/6:00:00 UTC
• Every 2 hours: OnCalendar=*-*-* 00/2:00:00 UTC

systemctl --user daemon-reload
systemctl --user enable --now myapp-triage.timer
systemctl --user enable --now myapp-implement.timer

# Enable lingering so timers run when you're not logged in
loginctl enable-linger $USER

# Verify timers are active
systemctl --user list-timers | grep myapp

The timer_prefix in autoloop.toml controls which timers autoloop status looks for. Name your timers with your app's prefix. It doesn't have to be "autoloop":

# If your timers are named myapp-triage.timer / myapp-implement.timer:
timer_prefix = "myapp"

# Default matches autoloop-triage.timer / autoloop-implement.timer:
# timer_prefix = "autoloop"

This supports multiple repos on the same VPS. Each repo has its own autoloop.toml with a distinct prefix.

Mobile workflow

Requires a Claude Code session running in tmux on your VPS. Start it once:

ssh your-vps
cd ~/your-project
tmux new -s claude
claude
# Detach: Ctrl+B then D

The session persists after you disconnect. Reconnect anytime with tmux attach.

Configure the autoloop MCP server by adding to your project's .mcp.json:

{
  "mcpServers": {
    "autoloop-mcp": {
      "command": "autoloop-mcp",
      "autoApprove": ["autoloop_status", "autoloop_triage", "autoloop_implement", "autoloop_fix_pr", "autoloop_review_pr"]
    }
  }
}

Runs started through the MCP tools (implement, triage, fix-pr, preflight) write their stdout and stderr to ~/.autoloop/<repo-hash>/mcp-runs/<UTC timestamp>-<command>.log; the tool's reply includes the path.

Install the MCP server on the VPS with uv tool install "autoloop[mcp] @ git+https://github.com/Sanctum-Origo-Systems/autoloop@<tag>", then restart the Claude Code session.

Enable remote control: In the Claude Code session on the VPS, run /login to authenticate, then enable remote connections in settings. This allows the Claude mobile app to send messages to the VPS session.

Two mobile apps, two roles:

• GitHub mobile app (iOS / Android): review diffs, approve, and merge PRs
• Claude mobile app (iOS / Android): tap Code at the bottom, select your VPS session from the list, and use the chat to invoke autoloop commands

Example workflow from your phone:

1. GitHub app: review and merge a PR
2. Claude app: "Implement the next ready issue". Autoloop picks the top issue and starts working
3. Claude app: "Check autoloop status". See progress, ready issue count, next timer
4. Claude app: "Fix PR 42". Rebases and resolves conflicts or failing checks

Configuration Reference

Field	Default	Description
repo	(required)	GitHub owner/repo (required)
triage_model	sonnet	Claude model for triage
impl_model	claude-opus-5-5	Claude model for implementation. Must be supported by the installed claude CLI (Opus 5.5 needs v2.1.310+)
review_model	(defaults to impl_model)	Claude model for semantic PR review. Falls back to impl_model when omitted
impl_timeout	900	Implementation timeout (seconds)
triage_timeout	90	Triage timeout (seconds)
test_timeout	120	Test command timeout (seconds)
pr_reviewer	(none)	GitHub username assigned to PRs
max_retries	3	Retry attempts per issue
max_story_points	3	Issues above this are decomposed
max_decomposition_depth	2	Max nesting depth before decomposition is blocked (needs-human)
verify_cmd	uv run pytest	Command to validate implementation
lint_command	uv run ruff check && uv run ruff format --check	Lint check command
timer_prefix	autoloop	Systemd timer prefix for status detection (use your app name, e.g. myapp)
protected_paths	["autoloop/"]	Paths the bot must never modify
test_gate_skip_types	["refactor", "docs", "chore"]	Issue types that skip the "must add test files" verification check

All fields can be overridden by environment variables (e.g. AUTOLOOP_IMPL_MODEL, AUTOLOOP_TIMEOUT).

Worktree Isolation

By default, autoloop runs in your project directory. When worktree isolation is enabled, implement, review-pr, and fix-pr operate in a persistent git worktree at ~/.autoloop/<id>/worktrees/<branch>, leaving your working tree untouched.

[implement]
isolation = "worktree"

Benefits:

• No session conflicts — autoloop and your interactive Claude Code session can run simultaneously
• Untracked local files (e.g. CLAUDE.local.md) don't leak into the builder's context
• Dependencies (.venv, node_modules) persist across runs — no reinstall per issue

The worktree is reset to a clean branch from origin/main before each issue. Session detection is skipped for commands that use the worktree since they never touch your tree.

With isolation on, implement's agent, verification, push and PR creation all run in the worktree, and the main checkout stays on main for the whole run. (It is still reset to a clean, up-to-date main at the start of each issue, so local main matches origin/main for branch comparisons.) autoloop doctor does a dry-run resolution of implement's working directory and fails if it is not under the worktree directory.

Start with isolation = "off" (default). Enable on one repo to validate, then default on once first-attempt rates hold.

Graduated Autonomy

The auto-merge gate controls when autoloop earns the right to merge PRs without human review. It evaluates three metrics against configurable thresholds:

Metric	Config Field	Default	Condition
First-attempt success rate	auto_merge_success_threshold	0.90	Must be strictly above
Human edit rate	auto_merge_edit_rate_threshold	0.05	Must be strictly below
Clean merge volume	auto_merge_volume_floor	10	Must be at or above

Promotion levels (auto_merge_promotion_level):

• "module" (default): Each module is evaluated independently against the thresholds. A module must build its own track record before auto-merge activates for it. Best for repos with enough PR volume that per-module counts reach the volume floor.
• "repo": All modules are evaluated together using aggregate repo-wide metrics. The gate shows a single repo-level verdict. Best for repos with low volume where per-module counts may never reach the volume floor.

Example: high-volume repo (module-level, strict thresholds)

auto_merge_success_threshold = 0.95
auto_merge_volume_floor = 20
auto_merge_edit_rate_threshold = 0.03
auto_merge_promotion_level = "module"

Example: low-volume repo (repo-level, lower volume floor)

auto_merge_success_threshold = 0.90
auto_merge_volume_floor = 5
auto_merge_edit_rate_threshold = 0.05
auto_merge_promotion_level = "repo"

Experimental: Decision Model (Jev)

View the shadow report (JEV.md) — agreement rates, confidence distribution by action zone, and whether Jev is ready to act.

Jev is an off-by-default probabilistic decision model that can shadow autoloop's triage and auto-merge decisions. In shadow mode it evaluates every decision point alongside the existing pipeline, logs its observations to autoloop/jev_decisions.jsonl (via the jev_log module), and changes nothing — no behavior is affected unless you explicitly opt in. If you don't use Jev, skip this section entirely.

Prerequisites: An OpenRouter API key with access to the Decisions API. Set the key in the environment variable named by api_key_env (default: OPENROUTER_API_KEY).

Config — add a [jev] block to autoloop.toml:

Field	Default	Description
mode	"off"	"off" — disabled (default). "shadow" — evaluate and log, no behavior change. "gate" — reserved for future use, not yet implemented
api_key_env	"OPENROUTER_API_KEY"	Environment variable holding the OpenRouter API key
model	"typesafe/jev-1.13"	Model to use for Jev evaluation requests
timeout_seconds	10	HTTP timeout for Jev evaluation requests
gate_low	0.15	Lower bound of the uncertain-probability band (probabilities inside the band trigger fallback)
gate_high	0.85	Upper bound of the uncertain-probability band

mode can also be set via the JEV_MODE environment variable.

Commands

Command	Description
autoloop init	Scaffold autoloop onto a new repo
autoloop doctor	Verify environment and configuration (--live adds a paid permission probe)
autoloop plan	Create issues from a spec file
autoloop triage	Triage untriaged issues
autoloop implement	Implement top ready issue
autoloop fix-pr	Fix a PR (rebase, resolve conflicts, fix checks)
autoloop review-pr	Review a PR (mutation gate + semantic review, no merge)
autoloop status	Show last run, ready issues, timers
autoloop auto-close-parent	Close parent when all sub-issues done
autoloop prune	Remove worktrees and local autoloop/* branches whose PRs are merged or closed; keeps dirty worktrees, open PRs, and unmerged commits (listed for human review). autoloop implement runs prune at the start of each run, and autoloop removes a worktree right after it auto-merges that PR. --dry-run reports what would be removed without removing anything
autoloop eval	Performance tracking: snapshots, trends, per-module breakdown (zero LLM cost)
autoloop version	Print installed version

License

Apache 2.0