Expanse
A NixOS-based server infrastructure system: plug a machine in, join it to a
cluster, and get replicated storage, load-balanced services, and a control
plane that keeps running when any single node doesn't — without hand-rolled
distributed-systems code where a proven one already exists (Raft, DRBD,
LVM, WireGuard, nftables). See .plan/PREMISE.md for the full vision and
.plan/ARCHITECTURE.md for how it's actually built.
The project site is expanseos.org.
What works today
- Install — bootable ISO, TUI or unattended install, declarative
btrfs+LVM partitioning, impermanent root (wiped every reboot except
/persist). docs/INSTALL.md, docs/HARDWARE.md.
- Clustering — mDNS discovery, HMAC join tokens, a Raft-replicated,
mTLS-secured control plane, generations with rollback, cordon/drain/remove
node lifecycle, witness nodes. Starts on a single node and grows as nodes
join.
docs/CLUSTERING.md.
- Storage — DRBD-replicated volumes on LVM thin pools: create, resize,
snapshot/restore, online repair, split-brain recovery, losing a node for
good.
docs/STORAGE.md.
- Blocks (services) — a scheduler that deploys, scales, rolling-updates,
and reschedules systemd-supervised workloads with anti-affinity,
singleton, and daemonset placement.
docs/BLOCKS.md.
- Database — a Postgres block with streaming replication and
automatic failover.
docs/DATABASE.md.
- MariaDB — a MariaDB over a replicated volume behind a VIP; every
committed row survives failover.
docs/MARIADB.md.
- Caddy — a web server and reverse proxy with automatic HTTPS; its
certificates survive failover.
docs/CADDY.md.
- HAProxy — a TCP and HTTP load balancer for services inside or outside
the cluster, as stateless replicas behind a VIP.
docs/HAPROXY.md.
- Forgejo — a Git forge with its repositories, issues and database on a
replicated volume behind a VIP; every acknowledged push survives failover.
docs/FORGEJO.md.
- Vaultwarden — a Bitwarden-compatible password manager with its vaults
and database on a replicated volume behind a VIP; every saved item survives
failover.
docs/VAULTWARDEN.md.
- Jellyfin — a media server with its library database and settings on a
replicated volume behind a VIP, streaming media the nodes share; every
acknowledged change survives failover.
docs/JELLYFIN.md.
- Uptime Kuma — status monitoring of websites and services, with its
monitors and history on a replicated volume behind a VIP; every acknowledged
monitor survives failover.
docs/UPTIME-KUMA.md.
- Immich — a photo and video library with its uploads, database and job
queue on a replicated volume behind a VIP; every acknowledged upload and
queued job survives failover.
docs/IMMICH.md.
- Scheduled backups — restic backups of block volumes to S3 or any restic
repository, taken from each volume's primary and following it through
failover.
docs/RESTIC-BACKUP.md.
- iSCSI — an LIO target over a replicated volume with VIP failover.
docs/ISCSI.md.
- NFS — an NFSv4.1 export (NFS-Ganesha) over a replicated volume; clients
reclaim their state on failover.
docs/NFS.md.
- S3 object storage — Garage over a replicated volume behind a VIP;
objects and keys survive failover.
docs/S3.md.
- Virtualized workloads — QEMU/KVM blocks with disks on replicated
volumes, restarting elsewhere on node loss.
docs/VMS.md.
- Pando — a guest image that runs the Pando app platform in a VM block;
Pando and its apps survive failover.
docs/PANDO.md.
- Networking — a WireGuard mesh, per-node nftables, and L4/L7 load
balancing with VIP failover.
docs/NETWORKING.md.
- Web UI — a browser-based cluster console (HTMX/SSE), OIDC login.
Deploy and edit any block, VMs included, from a form; turn storage on per node.
docs/WEB-UI.md.
- Backup/restore — restic-based backup of both cluster config and
volume data.
docs/BACKUP.md.
- Observability — Prometheus metrics, alert rules, node/volume/quorum
health.
docs/OBSERVABILITY.md.
- Security — a cluster CA with zero-downtime rotation, HMAC join
tokens, mTLS everywhere, OIDC login for the web UI. Secrets are
age-encrypted today; TPM sealing is a named, deliberately deferred
future feature (no TPM hardware has been available to validate it).
docs/SECURITY.md.
- Rolling upgrades — switch one node at a time with continuous
availability and zero acked-write loss.
docs/UPGRADE.md.
- Agent internals — the desired-state reconcile loop every node runs.
docs/AGENT.md.
Paused, not built: SMB failover (Samba deploy works; failover has an open
bug) and an on-prem LLM subsystem (no accelerator hardware available to
validate against yet). Both are independent of everything above and can
resume any time; nothing else depends on them.
Quickstart
# Build (or download) the installer ISO and write it to a USB stick.
nix build github:expanse/expanse#iso
sudo dd if=result/iso/*.iso of=/dev/sdX bs=4M status=progress conv=fsync
# Boot the target machine from it, then either:
expanse install --tui # interactive, or
# ...or an unattended install — see docs/INSTALL.md for both.
Minimum target hardware: 2 CPU cores, 2 GB RAM, a 20 GB disk — a decade-old
laptop is the design point, not an afterthought. Full walkthrough,
including forming and joining a cluster afterward:
docs/INSTALL.md → docs/CLUSTERING.md.
Documentation
Contributing / developing
nix develop # go, golangci-lint, protoc, qemu, everything else needed
make build && make test && make lint
See docs/DEVELOPING.md for the full workflow,
and .plan/ARCHITECTURE.md for the design principles (adopt proven data-plane
components; build the control plane) new code is held to.
Status
All 11 phases of .plan/ROADMAP.md have shipped. Every feature listed
above under "What works today" has NixOS VM-test coverage exercising real
node kills, partitions, and restarts — not just unit tests. The latest
release and its installer ISO are on the
releases page;
CHANGELOG.md records every release and its known issues, and
.plan/ARCHITECTURE.md §9 is the project's decision record.
How Expanse is made
Expanse is a joint human–AI project. Its code, tests and documentation were
built by a human maintainer working with AI coding assistants. The AI wrote
much of the code and prose. The maintainer directs the work, reviews it and
decides what ships.
- It is in the history. Most commits credit an AI co-author in a
Co-Authored-By trailer; git log shows which.
- Claims come with evidence. The VM-test coverage described under
Status exercises failures, not just the happy path, and
CHANGELOG.md records what is unverified. Real hardware,
for example, is still untested.
- Judge it on its record. Automated tests are no substitute for your own
evaluation; read the known issues before trusting Expanse with data you care
about.
See also How Expanse is made
on the project site.
License
Expanse is licensed under the Apache License, Version 2.0; see
NOTICE. Contributions are accepted under the same licence.
Vendored Go modules keep their own licences.
