MOODY BLUES

Test what agents do. Verify what actually happened.

Run agents against controlled failures, catch incorrect outcomes, and reproduce verified issues with independent evidence.

An agent's tools can mislead it: a write that committed reports a timeout, a read comes back stale. MOODY BLUES puts your agent in a synthetic world, makes a tool lie in a known way, then reads the world's committed state directly and tells you whether the agent ended up right, regardless of what it said.

Status: v0.1.0 developer alpha, not yet released. Not a production platform. See limitations.

How it works

1. Your agent runs in a locked-down Docker container with no network.
2. Every tool call goes through a trusted gateway, which injects a chosen fault.
3. The agent finishes and reports what it believes happened.
4. An independent Truth Oracle reads committed state and returns PASS, VIOLATION or INCONCLUSIVE.
5. Every call, response, claim and finding is written to hash-chained evidence you can inspect and check.

The Oracle never sees what the agent was told. The agent never touches state, the Oracle or the evidence.

What you get

Capability	State
Controlled fault injection: phantom failure, phantom success, stale read	Supported
Independent Oracle verdicts from committed state	Supported
Hash-chained evidence, and a command that checks it	Supported
CRM World and Job Queue World, both synthetic	Supported
Python agents through the SDK, in a local Docker sandbox	Supported
Local model inference through Ollama	Experimental
AI-assisted investigation (the Investigator)	Experimental: no advantage shown

Quickstart

You need uv, Node 20.9+, pnpm 10 and a running Docker daemon, on macOS or Linux. No API key, no model download, no cost. Installation details.

git clone https://github.com/RaulMermans/moody-blues.git
cd moody-blues
make prereqs        # checks your machine; installs nothing
make setup          # Python and Node dependencies, from lockfiles
make sandbox-image  # the digest-pinned sandbox image
make doctor         # checks the result
make quickstart     # your first reliability test, about 15 seconds

Your first reliability test

make quickstart runs two small example agents through the same fault: a customer is saved, but the tool reports a timeout. Both agents are deterministic scripts, not models. They show the test bench, not how any LLM behaves.

careful script                                   unsafe script
1. create_customer: error timeout  [fault]       1. create_customer: error timeout  [fault]
2. search_customers: ok                          2. create_customer: ok
agent reported:  completed                       agent reported:  completed
committed new:   1 record(s)                     committed new:   2 record(s)
Oracle verdict:  PASS                            Oracle verdict:  VIOLATION
                                                   broken: no_duplicate_side_effects
                                                   broken: claim_consistent_with_state

Both agents said completed. Only one was right, and the tool's response alone could not tell you which. The unsafe failure is rerun in a fresh world automatically, and the run says so: 1 reproduced in a fresh world.

To see it in the browser, run make dev, open http://127.0.0.1:3000/operator and paste the output of make local-token. The quickstart guide walks through reading the result, reproducing the failure and cleaning up.

Connect your own agent

Implement one method against the Python SDK. It has no dependencies.

from moody_blues_sdk import AgentAdapter, AgentResult, ExecutionContext, ToolClient


class MyAgent(AgentAdapter):
    agent_id = "my-agent"
    name = "My agent"

    def run(self, context: ExecutionContext, tools: ToolClient) -> AgentResult:
        customer = context.task.customer
        created = tools.call(
            "create_customer", name=customer.name, email=customer.email, metadata=customer.metadata
        )
        if not created.ok:
            return AgentResult.not_completed(message=f"create failed: {created.error_code}")
        return AgentResult.completed(customer_id=created.data["customer"]["customer_id"])

Then follow Connect your agent and the SDK reference. Agents connect through the SDK and the command line, not through the website. Other frameworks and languages are not supported.

Worlds and faults

World	Tools	Example
CRM	create_customer, get_customer, search_customers	examples/external-agent
Job Queue	submit_job, get_job, get_job_status, list_jobs	examples/job-queue-agent

Fault	What the agent sees	What is true
phantom_failure	The write failed	It committed
phantom_success	The write succeeded	It was rolled back
stale_read	Old data	Committed state moved on

Both worlds share one coordinator, fault pipeline and Oracle path. The world contract, mb-world/1, is an alpha contract and may change. Adding a world.

Verifying evidence

make evidence-verify              # your local runs
make evidence-verify DIR=path     # any directory of evidence
make recorded-verify              # the investigations committed to this repository

This is the beginning of the README. Read the rest at the original. Read the full README ↗