operational-problem-solver

Parallel worker teams · a consensus stability gate · an ELO tournament that ships the single best solution.

     

🌐 Live page  ·  Architecture  ·  Skill

A multi-team agentic engine for Claude Code. It spins up parallel worker teams to solve a technical problem, gates their output on a consensus stability threshold, then runs an ELO tournament to rank the surviving solutions head-to-head and ship the single best one.

                 [ Central Orchestrator ]
                /          |             \
   [ Intelligence ]   [ Execution ]   [ Quality Assurance ]
      Crow x2            Finch x4       Consensus + ELO
   context manifest   N solutions     gate -> tournament -> winner

Two halves, one repo:

• A Claude Code skill (.claude/skills/problem-solver/) — the in-session orchestration layer.
• A standalone Python engine (ops-solve) — the real work: parallel fan-out over the Anthropic Messages API, linters/tests via subprocess, and a deterministic ELO tournament.

---

Has this been done before?

The pieces are well-established; this exact packaged combination for Claude Code was not on the shelf when this was built. Honest prior art:

• Parallel Claude-agent orchestration already exists — claude-flow, claude_code_agent_farm, swarms, and the native Claude Code Task / Workflow tools. None ship a consensus-gated ELO tournament over competing code solutions as a packaged skill.
• The algorithms are proven research — Self-Consistency, AlphaCode (generate → filter → cluster → rank), CodeT, LLM-as-judge / MT-Bench, Chatbot Arena (Bradley-Terry / Elo), Mixture-of-Agents.
• ELO-ranking of agents/pipelines exists as tooling — RAGElo, CodeElo, llm-council. The closest end-to-end analog is AlphaEvolve (generate + evaluate + evolve) — but it is Gemini/Google and not packaged for Claude.

So this project is novel as an integration, not as research. It reuses known-good ELO / Bradley-Terry math and the AlphaCode / LLM-as-judge patterns.

---

⚠️ Cost — this is not free

Each worker and the judge are metered Claude API calls. The default fan-out is 18 calls per run: 2 Intelligence + 4 Execution + 6 judge pairs x 2 orders (each pair is judged both ways to cancel position bias). A consensus re-loop adds 4 more (22). 12 of the 18 go to the Opus judge, so the judge dominates the bill. The CLI prints the expected count before it starts. Judge calls grow with C(workers, 2) x rounds, so:

• --cheap runs workers on Haiku 4.5; the judge stays on Opus 4.8.
• Fewer --workers (3 workers: 3 pairs, 11 calls), fewer --rounds, --reloops 0.
• --concurrency caps calls in flight (default 8); lower it if you hit rate limits.
• Prompt caching is built in: all workers share one model + a cached Context Manifest prefix, and the judge caches the shared problem context across rounds.

The engine prints an estimated USD cost at the end of every run.

The offline test suite (pytest) exercises the ELO math, the consensus gate, and the full orchestrator loop with a mocked model — zero API spend.

---

Setup

git clone https://github.com/NAJEMWEHBE/operational-problem-solver
cd operational-problem-solver
python -m venv .venv; .venv\Scripts\Activate.ps1      # Windows PowerShell
pip install -e .
Copy-Item .env.example .env                            # then edit: ANTHROPIC_API_KEY=sk-ant-...
pytest -q                                              # offline logic tests, no API spend

macOS / Linux: source .venv/bin/activate and cp .env.example .env.

---

Usage

# Smallest real smoke test (cheap):
ops-solve "solve: write a Python is_prime(n) with a few tests" --workers 2 --cheap

# Gate consensus on real tests (EXECUTES the generated code — see Security):
ops-solve "solve: write a Python is_prime(n) with tests" --workers 4 --test-cmd "pytest -q {file}"

# Write the winning file somewhere:
ops-solve "solve: a CLI that reverses stdin" --out ./reverse.py

Options: --workers/-w, --intel/-i, --rounds/-r, --reloops, --concurrency/-c, --test-cmd, --out, --force (let --out overwrite), --cheap, --lang, --debug (full tracebacks). ops-solve --help lists them all.

여기까지가 README 앞부분입니다. 나머지는 원문에서 읽을 수 있습니다. README 전문 보기 ↗