Documentation · Workflow Gallery · Community · Roadmap · Contributing · Security
What is oxo-flow?
oxo-flow is a high-performance bioinformatics pipeline engine built in Rust. It compiles workflows into Directed Acyclic Graphs and orchestrates execution with native concurrency, per-rule environment isolation, and AI-powered assistance — all from a single binary.
🧬 Community: browse curated, rated, ready-to-run workflows — ports of popular nf-core & Snakemake pipelines, original designs, and community submissions — at oxo-flow-community.
- 🤖 AI Companion — Natural language pipeline generation, intelligent refinement, failure diagnosis, and results interpretation. Powered by Claude, OpenAI, DeepSeek, or local Ollama.
- 🔀 DAG engine — Automatic dependency resolution, topological ordering, and parallel execution with resource-aware scheduling (CPU, memory, GPU, disk) across local and cluster backends (SLURM, PBS, SGE, LSF)
- 📦 8 environment backends — conda, mamba, pixi, docker, singularity, venv, system, and HPC modules — with per-rule isolation
- ⚡ Rust performance — Fearless concurrency, zero-cost abstractions,
#![forbid(unsafe_code)] in every workspace crate
- 🌐 Professional Web UI — React 19 SPA with DAG visualization (React Flow + d3-dag), TOML editor (CodeMirror 6), and AI chat
- 🧩 VS Code extension — first-party
traitome.oxo-flow IDE for .oxoflow: schema-driven completion, background diagnostics, canonical formatting, one-click run / dry-run / graph / status / clean / resume / AI-generate (Open VSX + release VSIX)
- 📊 Built-in reporting — HTML/JSON/Markdown/PDF reports with execution summaries, failure diagnosis, resource metrics, and checkpoint-verified file manifests; QC metrics parsed from real tool outputs (fastp, flagstat, STAR, featureCounts, bcftools, kraken2), R-friendly TSV export (
--r-data), and a JSON report snapshot auto-written after every run
- 🔒 Security hardened — Shell injection prevention, path traversal protection, secret scanning, and per-IP rate limiting
- 🗄️ Checkpoint & resume — JSON-persisted execution state; resume interrupted workflows from the last completed rule
- 🚀 Three deployment modes — Personal workstation, team server with OAuth2, or HPC submit panel — same binary
Why oxo-flow?
| Feature | oxo-flow | Snakemake | Nextflow |
|---|
| Language | Rust — compiled, type-safe, #![forbid(unsafe_code)] in the core, AI, CLI, and web crates | Python | Groovy/JVM |
| Performance | Native binary, instant startup | Python + JIT overhead | JVM startup overhead |
| Workflow format | TOML (.oxoflow) — declarative, composable | Snakefile (Python DSL) | Nextflow DSL (Groovy) |
| Environment support | 8 backends — conda, mamba, pixi, docker, singularity, venv, system, modules — per-rule | conda, singularity, docker | conda, docker, singularity, modules |
| Web interface | Built-in React 19 SPA + REST API | External Snakemake-UI | Nextflow Tower (commercial) |
| Reporting | Built-in HTML/JSON/Markdown/PDF reports from checkpoint data (execution truth, failure diagnosis, file checksums) | Via MultiQC | Via Nextflow Tower |
| Cluster backends | SLURM, PBS, SGE, LSF | SLURM, PBS, SGE, LSF | SLURM, PBS, SGE, LSF, k8s |
| Security | Shell sanitization, path traversal prevention, rate limiting | Limited | Limited |
| AI Companion | Built-in — generate, refine, diagnose, interpret | Not built-in | Not built-in |
| Testing | 3,200+ tests (unit, integration, doc) | pytest-based | Varied |
Design Principles
oxo-flow is built on five principles:
- DAG is the fundamental abstraction — Every workflow is a directed acyclic graph. The engine constructs, validates, and executes DAGs with maximum parallelism.
- Environment isolation is non-negotiable — Each task runs in its own isolated environment via one of 8 backends.
- Reproducible by design — Config checksums, execution provenance, and container pinning guarantee identical outputs from identical inputs.
- Performance through Rust — Zero-cost abstractions and fearless concurrency for orchestrating thousands of concurrent tasks.
- Outcome-driven — The DAG engine's target-aware execution (
-t flag) computes the minimal rule set needed to produce specific deliverables.
Workflow Gallery
Learn oxo-flow incrementally with curated, validated example workflows — from a one-rule hello-world to production-grade pipelines. Every workflow passes oxo-flow validate and is tested in CI. See the Workflow Gallery for the full catalog with detailed explanations, DAG visualizations, and CLI output.
Quick Start
Install from pre-built binaries
Download the latest release for your platform from GitHub Releases:
# Linux (x86_64)
curl -LO https://github.com/Traitome/oxo-flow/releases/latest/download/oxo-flow-latest-x86_64-unknown-linux-gnu.tar.gz
tar xzf oxo-flow-latest-x86_64-unknown-linux-gnu.tar.gz
sudo mv oxo-flow /usr/local/bin/
# macOS (Apple Silicon)
curl -LO https://github.com/Traitome/oxo-flow/releases/latest/download/oxo-flow-latest-aarch64-apple-darwin.tar.gz
tar xzf oxo-flow-latest-aarch64-apple-darwin.tar.gz
sudo mv oxo-flow /usr/local/bin/
Install with cargo
cargo install oxo-flow-cli
Install with Conda
conda install -c bioconda oxo-flow-cli
Version caveat — the bioconda recipe can lag a freshly cut release
while the bot recipe update lands. Check conda search -c bioconda oxo-flow-cli for the currently delivered version; if it is behind the
release notes you need, install from the pre-built binaries above or
cargo install oxo-flow-cli (crates.io).
Run with Docker
Pre-built images are published to GitHub Container Registry on every release (and every push to main). Release images are multi-arch (linux/amd64 + linux/arm64, so Apple silicon and ARM servers pull a native image). :latest moves only after the release image passes a health smoke test; a rolling :<major.minor> tag (e.g. :0.19) tracks the newest patch of each minor line, and :main is a multi-arch dev build from source:
# Web UI at http://localhost:3000 (CLI is inside the same image)
docker run -d --name oxo-flow -p 3000:3000 ghcr.io/traitome/oxo-flow:latest
# Pin a release (or track a minor line)
docker run -d -p 3000:3000 ghcr.io/traitome/oxo-flow:0.25.0
docker run -d -p 3000:3000 ghcr.io/traitome/oxo-flow:0.25
# CLI one-shot usage (workflow files mounted from the host)
docker run --rm -v "$PWD:/work" -w /work ghcr.io/traitome/oxo-flow:latest \
oxo-flow run my-pipeline.oxoflow
Health endpoint: GET /api/health returns "status":"ok" when the database is reachable.
Container configuration is env-only: the image sets OXO_FLOW_MODE=team, OXO_FLOW_HOST=0.0.0.0, and OXO_FLOW_PORT=3000 as defaults, and the CMD carries no CLI flags — so docker run -e OXO_FLOW_PORT=9090 … changes the actual bind port (the built-in healthcheck follows it). Note the published image is built without the postgres compile feature: pointing DATABASE_URL at a postgres:// URL makes the container exit at startup (see the docker-compose.yml comment for how to build a feature-enabled image).
Build from source
git clone https://github.com/Traitome/oxo-flow.git
cd oxo-flow
cargo build --release --workspace
# Build the web SPA so `oxo-flow serve` has assets to serve
# (the committed index.html references gitignored build output):
make frontend-build # or: cd frontend && npm install && npm run build
# Install the CLI to your local cargo bin directory:
cargo install --path crates/oxo-flow-cli
# Binaries are in target/release/
# - oxo-flow (CLI, includes `oxo-flow serve` for web UI)
# - oxo-flow-web (Standalone web server)
Skipping the frontend build is safe: oxo-flow serve detects the missing
SPA assets and shows an actionable "SPA not built" page instead of a blank
screen.
First workflow
# Create a new pipeline project (creates my-pipeline/ directory and my-pipeline.oxoflow)
oxo-flow init my-pipeline
cd my-pipeline
# Validate the workflow
oxo-flow validate my-pipeline.oxoflow
# Preview execution plan
oxo-flow dry-run my-pipeline.oxoflow
# Execute with 8 parallel jobs
oxo-flow run my-pipeline.oxoflow -j 8
# Visualize the DAG (use -f dot for Graphviz DOT output)
oxo-flow graph my-pipeline.oxoflow -f dot > dag.dot
dot -Tpng dag.dot -o dag.png
# Export a transit-map-style diagram via nf-metro
oxo-flow graph my-pipeline.oxoflow -f metro -o pipeline.mmd
# Render locally (requires nf-metro) or paste the .mmd content into the
# online playground: https://seqeralabs.github.io/nf-metro/latest/playground/
nf-metro render pipeline.mmd -o pipeline.svg
# Generate an HTML report
oxo-flow report my-pipeline.oxoflow -f html -o report.html
Cluster / HPC
oxo-flow supports job submission to HPC cluster schedulers including SLURM, PBS/PBS Pro, SGE/UGE, and LSF. The oxo-flow cluster subcommand manages the full submission lifecycle.
# Submit a workflow to a SLURM cluster
oxo-flow cluster submit workflow.oxoflow --backend slurm --queue short -o jobs/
# Check submission status (no job ids: lists the user's jobs; --backend optional)
oxo-flow cluster status --backend slurm <job-id>
oxo-flow cluster status
# Cancel a submitted job
oxo-flow cluster cancel --backend slurm <job-id>
Supported backends: slurm, pbs, sge, lsf. Cluster submissions are configured inline via oxo-flow cluster submit flags.
Workflows can also carry reusable config supplements (e.g. per-cluster thread
and memory defaults) in profiles/<NAME>.toml next to the workflow:
# Run a workflow with a config supplement profile
oxo-flow run workflow.oxoflow --profile slurm
Workflow Format (.oxoflow)
oxo-flow uses a TOML-based workflow format that is human-readable, composable, and declarative:
[workflow]
name = "variant-calling"
version = "1.0.0"
[config]
reference = "/data/ref/GRCh38.fa"
[[rules]]
name = "fastp"
input = ["raw/{sample}_R1.fastq.gz", "raw/{sample}_R2.fastq.gz"]
output = ["trimmed/{sample}_R1.fastq.gz", "trimmed/{sample}_R2.fastq.gz"]
threads = 8
shell = "fastp -i {input[0]} -I {input[1]} -o {output[0]} -O {output[1]}"
[rules.environment]
conda = "envs/fastp.yaml"
[[rules]]
name = "bwa_align"
input = ["trimmed/{sample}_R1.fastq.gz", "trimmed/{sample}_R2.fastq.gz"]
output = ["aligned/{sample}.bam"]
threads = 16
memory = "32G"
shell = "bwa-mem2 mem -t {threads} {config.reference} {input[0]} {input[1]} | samtools sort -o {output[0]}"
[rules.environment]
docker = "biocontainers/bwa-mem2:2.2.1"
Wildcards like {sample} expand automatically via input file discovery. Features include reference directory conventions, environment groups, optional rules, and directory inputs. See the full Workflow Format Specification for details.
Editor Support (VS Code)
The first-party oxo-flow Pipeline extension (traitome.oxo-flow) makes VS Code a full .oxoflow IDE — and it is released in lockstep with the engine (VSIX with checksum attached to every GitHub release):
Note — the extension is published to both marketplaces: Open VSX (searched by default by VSCodium, Cursor, Windsurf and most forks) and the VS Code Marketplace (stock VS Code, live since v0.25.0). Install from either, or grab the release VSIX offline:
codium --install-extension traitome.oxo-flow # Open VSX (forks)
code --install-extension oxo-flow-vscode-v<version>.vsix # offline VSIX (any VS Code)
It provides [[rules]]-aware syntax highlighting (wildcards and {input[0]} placeholders), completion and hover docs generated from the canonical workflow JSON Schema, background validate/lint diagnostics anchored to the failing rule, canonical formatting, and one-click run / dry-run / graph / resume / AI-generate commands. See the VS Code Extension reference, or Editor Setup for other editors (Zed, Helix, Neovim, JetBrains, …).
CLI Commands
The oxo-flow binary provides 31 subcommands covering the complete workflow lifecycle. See the full CLI Reference for details.
| Category | Commands |
|---|
| Execution | run, resume, dry-run (alias plan), test, batch |
| Development | init, validate, format, lint, debug, template |
| Inspection | graph, info, report, status, config (alias check), diff, provenance, schema, verify-data |
| Environment | env, export, clean, touch |
| Deployment | serve, cluster, publish, pull |
| AI & System | ai, completions, license |
See the full CLI Reference for detailed usage of each subcommand.
Web API
The oxo-flow serve command starts an axum-powered REST server with 100+ endpoints across 9 domains (workflow, execution, dag, AI, auth, collaboration, observability, chat, clusters). Full API reference at the Web API reference (GET /api/openapi.json serves the live OpenAPI 3.1 spec).
oxo-flow is organized as a Cargo workspace with four crates, plus a fifth
(oxo-flow-desktop) that lives in crates/ but is excluded from the
workspace so headless builds never need GUI toolchains:
oxo-flow/
├── crates/
│ ├── oxo-flow-core/ # Core library: DAG engine, executor, environment mgmt,
│ │ # config parsing, scheduler, wildcard expansion, reporting
│ ├── oxo-flow-ai/ # AI companion: provider abstraction, skill system, agents
│ ├── oxo-flow-cli/ # CLI binary ("oxo-flow") — Clap-based, 31 subcommands
│ ├── oxo-flow-web/ # Web server ("oxo-flow-web") — axum REST API + frontend
│ └── oxo-flow-desktop/ # Native desktop shell ("oxo-flow-desktop", wry + tao) —
│ # excluded from the workspace (own Cargo.toml)
├── examples/ # Example .oxoflow workflows
├── tests/ # Integration tests
└── docs/ # Documentation (MkDocs)
| Crate | Type | Binary | License |
|---|
oxo-flow-core | Library | — | Apache-2.0 |
oxo-flow-ai | Library | — | Apache-2.0 |
oxo-flow-cli | Binary | oxo-flow | Apache-2.0 |
oxo-flow-web | Binary | oxo-flow-web | Dual Academic / Commercial |
oxo-flow-desktop | Binary | oxo-flow-desktop | Apache-2.0 |
Key modules
| Module | Crate | Responsibility |
|---|
dag.rs | core | DAG construction, validation, topological sort |
executor/ | core | Task execution (local, cluster, cloud), checkpointing, staging |
environment.rs | core | Environment management (conda, mamba, pixi, docker, singularity, venv, system, modules) |
config/ | core | Workflow configuration and .oxoflow file parsing |
rule.rs | core | Rule/step definitions with inputs, outputs, shell, resources |
scheduler.rs | core | Job scheduling with resource constraints |
wildcard.rs | core | Wildcard pattern expansion ({sample}, {chr}, etc.) |
report.rs | core | Modular report generation (HTML, JSON, PDF via wkhtmltopdf) |
format.rs | core | Workflow formatting, linting, and 30+ diagnostic patterns |
plugin.rs | core | Plugin system for MCP servers and custom tools |
ai_provider.rs | web | Multi-provider AI abstraction (Claude, OpenAI, DeepSeek, Ollama) |
domains/ai/ | web | AI translation, chat SSE streaming, agent orchestration |
domains/execution/ | web | Run lifecycle, diagnostics, pause/resume, retry |
domains/workflow/ | web | Pipeline CRUD, validation, DAG building, templating |
provider.rs | ai | Multi-provider AI client abstraction with auto-detection |
skill.rs | ai | Skill registry and discovery system |
agent/ | ai | Agent orchestration and tool-calling framework |
mcp.rs | ai | MCP (Model Context Protocol) server integration |
Three-Mode Deployment
oxo-flow serve starts the web interface with an embedded REST API and React SPA. Run it without arguments for local experimentation:
# Mode 1: Personal workstation (default) — SQLite, localhost:8080, no auth
oxo-flow serve
# → Open http://127.0.0.1:8080 in your browser
# Mode 2: Team server — multi-user (env-var passwords + optional ORCID/GitHub OAuth2)
oxo-flow serve --mode team --host 0.0.0.0
# Mode 3: HPC submit panel — the scheduler (SLURM/PBS/SGE/LSF) is probed at
# startup when the server itself runs on a cluster login node
oxo-flow serve --mode hpc
Serve flags: --mode (personal | team | hpc), --host (default
127.0.0.1), -p/--port (default 8080), --base-path, and --open
(open the browser on startup). Each flag also has an OXO_FLOW_*
environment-variable form, plus a platform config file layer — see
Deployment Modes.
PostgreSQL: both oxo-flow serve and the standalone oxo-flow-web
binary read the DATABASE_URL environment variable — a postgres:// URL
(on a build with the postgres feature) enables the PostgreSQL backend for
the library/AI/auth domains; the default (sqlite://oxo-flow.db in the
working directory) stays SQLite. Note that the published Docker image is
built WITHOUT the postgres feature — rebuild with
cargo build --features postgres to enable it. Run execution stays
SQLite-only either way: on a PostgreSQL server every /api/runs* endpoint
answers 503 RUNS_REQUIRE_SQLITE.
Documentation
Full documentation is available at traitome.github.io/oxo-flow/latest/.
📖 Documentation Quick Links
MkDocs source lives under docs/guide/src/.
Development
# Build all workspace crates
cargo build --workspace
# Run all tests (unit + integration)
cargo test --workspace
# Run the full CI suite (fmt + clippy + build + test + schema-drift + version-check + audit + frontend lint)
make ci
# Individual CI steps
cargo fmt -- --check # Check formatting
cargo clippy --workspace --all-targets -- -D warnings # Lint (zero warnings)
cargo build --workspace # Compile
cargo test --workspace # Test
# Format code
cargo fmt
A bare cargo build at the repo root builds only the root oxo-flow
library package (the integration-test crate) — not the CLI or web
binaries. Pass --workspace whenever you want target/*/oxo-flow and
target/*/oxo-flow-web.
Tech stack
License
This project uses a split licensing model:
| Crate | License | Details |
|---|
oxo-flow-core | Apache-2.0 | Free and open-source |
oxo-flow-ai | Apache-2.0 | Free and open-source |
oxo-flow-cli | Apache-2.0 | Free and open-source |
oxo-flow-web | Academic / Commercial | Free for academic and non-commercial use; commercial use requires a separate license |
The core library, AI companion, and CLI are licensed under the Apache License 2.0 — you are free to use, modify, and distribute them without restriction.
The web interface (oxo-flow-web) is available under a dual license: free for academic and non-commercial use under the Academic License, and requiring a commercial license for commercial deployments. See LICENSE-ACADEMIC and LICENSE-COMMERCIAL for details.
Contributing
Contributions are welcome! Please see:
Before submitting a PR, ensure all checks pass:
make ci
Citing
If you use oxo-flow in academic research, please cite:
Shixiang Wang, oxo-flow: compiled, memory-safe bioinformatics workflow orchestration, bioRxiv, 2026, https://doi.org/10.64898/2026.06.11.731578
Jia Ding, Yun Peng, Ruochen Wei, Boquan Wang, Jian-Guo Zhou, Shixiang Wang, BLIT: an R package for seamless integration of command-line bioinformatics tool universe, Bioinformatics Advances, Volume 6, Issue 1, 2026, vbag088, https://doi.org/10.1093/bioadv/vbag088
Community
🧪 Real-World Feedback
oxo-flow aims to work reliably across diverse computing environments — laptops, HPC clusters, GPU nodes. We cannot replicate every deployment scenario in CI. If you use oxo-flow, please share your experience (success or failure) as a GitHub Issue with the prefix [Real-World Testing]. Your feedback directly shapes our priorities.
Additional Resources
Built with 🧬 by Traitome
Contributors
Thanks goes to these wonderful people:
Run make contributors to refresh the contributor list from git history.
