Five Qwen3.8-27B agents beat Fable 5 on LiveCodeBench Hard, as a Memdoor workflow.
GVS5H's harness rebuilt with the
authors' exact prompts and loop. The first ten problems of their list, five
passes each: a mean of 9.8 of 10. The authors' own harness scored 9.6 and
Fable 5 in a single call 8.0, the gap the authors measured on these ten
(across their full 100: 92.4 and 90.4). All 50 runs are published: every
request, reply, program and grader output. The workflow is YAML and names no
model. Point it at yours, or at a local one with memdoor connect local
(janis, the runtime we recommend; Ollama, llama.cpp, vLLM and LM Studio
work too; how to run it on an open-weight model,
no key).
examples/workflows/team-solve · the result page · the HN thread
Memdoor is a coding agent and workflow runner. Give the agent a task to read
files, edit code and run checks. For repeatable work, a workflow coordinates
checked steps, parallel tasks and approval gates. Run it with a local model or
a provider on your key.
On this machine:
curl -fsSL https://memdoor.ai/install.sh | bash
cd your-project && memdoor # sets the machine up, then a key, a sign-in or a local model
# no key: memdoor connect local # janis and an open-weight model on this machine — https://memdoor.ai/docs/open-weight
memdoor cron add --workflow team-backlog "0 2 * * *" # one open issue a night, a pull request when its Check passed
The install or the first run did not finish: open an Install issue.
On a box reachable by ssh, nothing installed there:
curl -fsSL https://memdoor.ai/install.sh | bash # here, once
memdoor --gateway ssh://you@box # installs memdoor on the box, starts its gateway, opens the window on it
memdoor --gateway ssh://you@box connect openrouter # your key, kept on the box; every turn there runs on it
memdoor --gateway ssh://you@box run "fix the failing test" # the same box from a script or CI
Decision model. Jev (TypeSafe) answers a typed question about a turn
with a calibrated probability in under half a second. Judged reads: search,
file and log reads return what counts. Per-turn toolbox: only the tools the
turn needs are sent. Stall detection: a turn that stops making progress ends.
44 paired runs, decisions off and on: 22 of 22 passed each way; a codebase
question 54,661 to 14,367 input tokens (−74%); an edit with a test and ten
one-file fixes read the same files either way (within ±11%)
(docs/features/DECIDE.md).
Workflows, the deterministic harness. Say the steps; the coder writes one
YAML per step. mario runs the graph.
Independent steps run in parallel. A step is done when its check passes. A
gate waits for approval with the diff. A failed run resumes at the failed
step. Started by hand, from CI or on local cron
(docs/features/WORKFLOWS.md). Workflows, not
skills: the write-up.
Examples, each run before it was added:
examples/workflows/. The nightly grade step,
examples/workflows/nightly-rule/steps/grade.yaml:
type: agent
prompt: |
Today's partition is {{.partition}}.
Read rules/{{.partition}}.md and grade today's trading rule: assess it
briefly (clarity, risk profile, plausibility) and write a JSON object
{"score": <0-100 integer>, "verdict": "<one sentence>"} into
grades/{{.partition}}.json (create grades/ if needed).
requires:
- table_pattern: build
target:
# run by the engine, not by the agent: the grade must exist and be newer than step 1's start stamp
command: test "grades/{{.partition}}.json" -nt "rules/{{.partition}}.start" && test -s "grades/{{.partition}}.json"
timeout: 10m
The review loop recorded (findings verified by running, a gate, the fixes
applied): memdoor.ai/workflows.
In the window
| Command | What it does |
|---|
/model | The agent's ladder, cheapest rung first. Pin any model of your providers. List prices |
/connect | Add a provider: OpenRouter, Anthropic, OpenAI, Gemini, DeepSeek, Baseten, Groq, xAI, any OpenAI-compatible endpoint, a company AI gateway. /connect local: janis on this machine or the box, or Ollama, llama.cpp, LM Studio, vLLM; no key. /connect chatgpt: a ChatGPT Plus/Pro plan, no key |
/w | The project's workflows and runs. /w <name> [the task] runs one (/w team-solve <the task>), enter opens a run's graph, a approves a gate |
/usage | This month on your key, per model, and what the decision model kept out of the bill |
/remote | This conversation on your phone: a link and a QR code, the terminal itself, end-to-end encrypted |
/remote view | A watch-only link: live, no typing, no account, up to 200 viewers. ?live=1 is the terminal alone, for a stream or OBS |
/files or ctrl+f | The project's files, the ones this conversation read or changed first, each with what happened to it. Enter previews, ctrl+d diffs, ctrl+e opens in $EDITOR, tab puts @path in the prompt |
/graph or ctrl+g | The turn as a graph: the reads, the edits that depended on them, the runs that depended on the edits, the failing check in red |
/mcp | MCP servers from a URL, a command or a .mcp.json snippet |
/help | The rest |
Esc interrupts. ctrl+o expands tool frames. Shift+Tab switches the mode,
auto or team. ctrl+t sets the reasoning effort. @path mentions a file. MEMDOOR_APPROVE=changes asks before every
command and write.
This is the beginning of the README. Read the rest at the original. Read the full README ↗
