operational-problem-solver
Parallel worker teams · a consensus stability gate · an ELO tournament that ships the single best solution.
🌐 Live page · Architecture · Skill
A multi-team agentic engine for Claude Code. It spins up parallel worker teams
to solve a technical problem, gates their output on a consensus stability
threshold, then runs an ELO tournament to rank the surviving solutions
head-to-head and ship the single best one.
[ Central Orchestrator ]
/ | \
[ Intelligence ] [ Execution ] [ Quality Assurance ]
Crow x2 Finch x4 Consensus + ELO
context manifest N solutions gate -> tournament -> winner
Two halves, one repo:
- A Claude Code skill (
.claude/skills/problem-solver/) — the in-session
orchestration layer.
- A standalone Python engine (
ops-solve) — the real work: parallel fan-out
over the Anthropic Messages API, linters/tests via subprocess, and a
deterministic ELO tournament.
Has this been done before?
The pieces are well-established; this exact packaged combination for
Claude Code was not on the shelf when this was built. Honest prior art:
- Parallel Claude-agent orchestration already exists —
claude-flow,
claude_code_agent_farm,
swarms, and the native Claude Code Task /
Workflow tools. None ship a consensus-gated ELO tournament over competing code
solutions as a packaged skill.
- The algorithms are proven research —
Self-Consistency,
AlphaCode (generate → filter → cluster → rank),
CodeT,
LLM-as-judge / MT-Bench,
Chatbot Arena (Bradley-Terry / Elo),
Mixture-of-Agents.
- ELO-ranking of agents/pipelines exists as tooling —
RAGElo,
CodeElo,
llm-council. The closest end-to-end
analog is AlphaEvolve
(generate + evaluate + evolve) — but it is Gemini/Google and not packaged for Claude.
So this project is novel as an integration, not as research. It reuses
known-good ELO / Bradley-Terry math and the AlphaCode / LLM-as-judge patterns.
⚠️ Cost — this is not free
Each worker and the judge are metered Claude API calls. The default fan-out
is 18 calls per run: 2 Intelligence + 4 Execution + 6 judge pairs x 2 orders
(each pair is judged both ways to cancel position bias). A consensus re-loop adds
4 more (22). 12 of the 18 go to the Opus judge, so the judge dominates the bill.
The CLI prints the expected count before it starts. Judge calls grow with
C(workers, 2) x rounds, so:
--cheap runs workers on Haiku 4.5; the judge stays on Opus 4.8.- Fewer
--workers (3 workers: 3 pairs, 11 calls), fewer --rounds, --reloops 0.
--concurrency caps calls in flight (default 8); lower it if you hit rate limits.- Prompt caching is built in: all workers share one model + a cached Context
Manifest prefix, and the judge caches the shared problem context across rounds.
The engine prints an estimated USD cost at the end of every run.
The offline test suite (pytest) exercises the ELO math, the consensus gate, and
the full orchestrator loop with a mocked model — zero API spend.
Setup
git clone https://github.com/NAJEMWEHBE/operational-problem-solver
cd operational-problem-solver
python -m venv .venv; .venv\Scripts\Activate.ps1 # Windows PowerShell
pip install -e .
Copy-Item .env.example .env # then edit: ANTHROPIC_API_KEY=sk-ant-...
pytest -q # offline logic tests, no API spend
macOS / Linux: source .venv/bin/activate and cp .env.example .env.
Usage
# Smallest real smoke test (cheap):
ops-solve "solve: write a Python is_prime(n) with a few tests" --workers 2 --cheap
# Gate consensus on real tests (EXECUTES the generated code — see Security):
ops-solve "solve: write a Python is_prime(n) with tests" --workers 4 --test-cmd "pytest -q {file}"
# Write the winning file somewhere:
ops-solve "solve: a CLI that reverses stdin" --out ./reverse.py
Options: --workers/-w, --intel/-i, --rounds/-r, --reloops, --concurrency/-c,
--test-cmd, --out, --force (let --out overwrite), --cheap, --lang,
--debug (full tracebacks). ops-solve --help lists them all.
This is the beginning of the README. Read the rest at the original. Read the full README ↗
