mdspec

Executable Markdown Testing.

Your docs are your tests. Write CLI commands and expected output in markdown code fences — mdspec runs them with persistent shell context, pattern matching, and snapshot updates.

Quick Start

Install from npm; a git install resolves the TypeScript source and runs only under Bun.

Requires Bun >=1.0 or Node.js >=18. PTY testing (cmd= attribute) on Node.js requires node-pty.

bun add -d mdspec

Write a test (example.spec.md):

# My CLI

```console
$ echo "Hello, mdspec!"
Hello, mdspec!
```

```console
$ date +"%Y"
/\d{4}/
```

Run it:

mdspec example.spec.md

When output changes, update snapshots in place:

mdspec --update example.spec.md

Features

Pattern Matching

Wildcards, regex, and named captures for dynamic output:

```console
$ uuidgen
{{id:/[0-9A-F-]{36}/}}
```

Ellipsis wildcards ([...] or ...) match any text inline or across lines.

Persistent Context

Shell state carries across separate code fences — env vars, working directory, functions all persist within a file:

```console
$ export DB="postgres://localhost/myapp"
$ createdb myapp
```

```console
$ my-cli migrate --db "$DB"
Applied 3 migrations.
```

Shell defaults

Ordinary Bash test commands run with pipefail on and errexit off. A failed upstream stage makes the pipeline fail, and expected nonzero exits remain assertable. Use standard Bash syntax to opt out for one command:

```console
$ false | cat
[1]
$ set +o pipefail; false | cat
[0]
$ false | cat
[1]
```

Each command runs in a fresh Bash, so the opt-out does not carry into the next command. These defaults apply to Bash execution, not code inside an embedded JavaScript REPL or a custom plugin.

Fixture Lifecycle

Set up once from clean shell state and guarantee teardown, even after a failed example:

```beforeAll reset
export FIXTURE="$(mktemp -d)"
cd "$FIXTURE"
```

```afterAll
rm -rf "$FIXTURE"
```

Lifecycle fence bodies are raw shell and do not count as executable examples.

Plugins

Replace shell execution with in-process TypeScript plugins for dramatically faster test runs:

---
mdspec:
  plugin: ./my-plugin.ts
---

REPL Testing

Test interactive REPLs with persistent subprocess mode:

```console cmd="node -i"
$ 1 + 1
2
$ 'hello'.toUpperCase()
'HELLO'
```

Test Runner Integration

Runs inside Vitest or Bun — same config, reporters, and CI you already have:

import { registerMdTests } from "mdspec/vitest"
await registerMdTests("tests/**/*.spec.md")

CLI

mdspec <patterns...>            # Run tests
mdspec --update tests/*.spec.md # Update snapshots
mdspec --dots tests/*.spec.md   # Compact dots reporter
mdspec --tap tests/*.spec.md    # TAP output

How mdspec Is Different

Most CLI testing tools were built for a specific language ecosystem or require custom file formats. mdspec takes a different approach:

Standard markdown, not custom formats. Tests are .md files that render on GitHub, in docs sites, and in editors. No .t files, no TOML manifests, no new syntax to learn. If you already write console examples in your README, you're halfway there.

JavaScript-native with in-process plugins. For JS/TS CLIs, plugins bypass the shell entirely — your test calls your code as a function, not a subprocess. This is unique to mdspec and makes tests dramatically faster.

Integrated, not standalone. mdspec runs inside Vitest or Bun's test runner with the same config, reporters, and CI you already have. No separate tool to install, no parallel test pipeline to maintain.

	mdspec	Cram	trycmd	shelltestrunner	doctest
Ecosystem	JS/TS	Python	Rust	Haskell	Python
Format	Markdown	.t files	TOML + .md	Custom	Docstrings
In-process	Plugins	No	No	No	Yes
Test runner	Vitest, Bun	Standalone	cargo test	Standalone	unittest
Persistent context	Across fences	Per block	Per file	No	No
Named captures	Yes	No	No	No	No

Documentation

Full documentation: beorn.github.io/mdspec

Security

mdspec executes shell commands from markdown blocks. Do not run it on untrusted content.

License

MIT