OpenHydra

---

OpenHydra is a peer-to-peer protocol for distributed AI inference. It does not run a model itself. Instead it sits in front of whatever inference engine you already run — Ollama, vLLM, LM Studio, llama.cpp, or any OpenAI-compatible server — and joins a global swarm that routes each request to a peer that has the requested model, streams the completion back, and settles a co-signed receipt. No central server. No API keys. No subscription. Think BitTorrent, for AI inference.

The whole thing is a single Rust binary, openhydra-agent. There is no Python in any role.

Why OpenHydra?

• Bring your own engine. Already running Ollama, vLLM, LM Studio, or llama.cpp? Join with a thin adapter and keep your stack — any Mac, NVIDIA, or AMD GPU, any platform. Requests run on the provider at native local speed.
• No central server. Peers discover each other over a Kademlia DHT + mDNS, with AutoNAT, UPnP, DCUtR hole-punching, and Circuit Relay v2 fallback for NAT'd nodes. The network is the computer.
• Free, by reciprocity. Give-to-get: serve inference to build standing when you consume. No tokens, no crypto, no monthly bill.
• OpenAI-compatible. The consumer gateway speaks the OpenAI Chat Completions API (streaming and non-streaming), so existing SDKs and tools point at it unchanged.
• Keys stay put. Each node's ed25519 identity key never leaves the Rust daemon — receipts are co-signed in-process; only signatures cross the wire.

---

How it works

Two roles, one binary:

• provide — joins the swarm, detects the models your local engine serves, advertises their canonical ids on the DHT, and serves inbound inference by proxying to that engine. It never runs a model itself.
• serve — the consumer front door: an OpenAI-compatible HTTP/SSE gateway. For each request it discovers a provider for the requested model, streams the completion back over libp2p, and settles a co-signed receipt at EOS.

A request never touches a coordinator: the gateway talks to the provider directly (or through a relay when both sides are NAT'd). Throughput is your engine's native speed plus the routing/transport overhead.

---

Quick Start

To share an engine you need an inference engine running locally (Ollama, vLLM, LM Studio, or llama.cpp). To only consume from the network, no engine is required.

Install

Prebuilt binary (recommended)

One line, no toolchain — installs the latest openhydra-agent for your platform:

curl -fsSL https://openhydra.co/install.sh | sh

It drops the binary in /usr/local/bin (or ~/.local/bin if that isn't writable). Prefer to grab it yourself? Download the right asset from the latest release, chmod +x, and put it on your PATH:

Platform	Asset
macOS (Apple Silicon)	openhydra-agent-aarch64-apple-darwin
Linux (x86-64)	openhydra-agent-x86_64-unknown-linux-gnu
Windows (x86-64)	openhydra-agent-x86_64-pc-windows-msvc.exe

The binaries are not yet code-signed. On macOS, if Gatekeeper blocks the binary, clear the quarantine flag: xattr -d com.apple.quarantine ./openhydra-agent. (Intel Macs and Linux arm64 have no prebuilt binary yet — build from source.)

Desktop app

Prefer a GUI? Grab the installer for your OS from the latest release — .dmg (macOS), -setup.exe / .msi (Windows), .AppImage / .deb (Linux) — or use the one-click links at openhydra.co.

Build from source

Needs a stable Rust toolchain. The whole workspace is pure Rust — no Python toolchain is required.

git clone https://github.com/samtroberts/openhydra && cd openhydra
cargo build --release -p openhydra-agent
# → ./target/release/openhydra-agent

Share your engine (provider)

Start your engine, then point a provider at it. Ollama is the default:

# Ollama on its standard port (11434)
./target/release/openhydra-agent provide --engine-kind ollama

# vLLM / LM Studio / llama.cpp / any OpenAI-compatible server
./target/release/openhydra-agent provide --engine-kind vllm       --engine http://127.0.0.1:8000
./target/release/openhydra-agent provide --engine-kind lm-studio  --engine http://127.0.0.1:1234
./target/release/openhydra-agent provide --engine-kind llama-cpp  --engine http://127.0.0.1:8080
./target/release/openhydra-agent provide --engine-kind openai     --engine http://127.0.0.1:8000

# Exo MLX cluster (port 52415)
./target/release/openhydra-agent provide --engine-kind exo        --engine http://127.0.0.1:52415

# ComfyUI for image / audio / video (port 8188); add --comfyui-workflow-dir <dir> for BYO workflows
./target/release/openhydra-agent provide --engine-kind comfyui    --engine http://127.0.0.1:8188

Useful flags: --db <path> persists the receipt ledger (redb) across restarts; --bootstrap <multiaddr> dials a known peer/relay when mDNS can't reach the swarm; --reannounce-secs tunes the DHT re-announce interval (default 120s).

Consume (gateway)

Run the OpenAI-compatible gateway on another machine (or the same one, second process):

./target/release/openhydra-agent serve            # binds 127.0.0.1:16527
# optional: require a key on /v1/* —
#   --api-key <key>   or   OPENHYDRA_API_KEY=<key>

Then point any OpenAI client at it:

curl http://127.0.0.1:16527/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "llama3.2",
    "messages": [{"role": "user", "content": "Write a haiku about peer-to-peer AI."}],
    "stream": true
  }'

The gateway also serves GET /v1/models (model ids discovered on the swarm) and GET /health.

The gateway checks each provider's billed token count against the model's own tokenizer. Which tokenizer that is comes from a signed index (built in, refreshed daily from dl.openhydra.co); the tokenizer file itself is downloaded from huggingface.co the first time a listed model is used, hash-checked, and cached in ~/.openhydra/tokenizers. --no-tokenizer-downloads (desktop: Settings → Network → "Check billed token counts") turns the download off.

---

Supported engines

--engine-kind	Engine	Default port	Detection
ollama	Ollama (native /api/*)	11434	/api/tags — full canonical ids
vllm	vLLM (OpenAI /v1/*)	8000	/v1/models
lm-studio	LM Studio (OpenAI /v1/*)	1234	/v1/models
llama-cpp	llama-server (/props)	8080	/props → GGUF canonical id
exo	Exo MLX cluster	52415	/state — only placed-and-ready models (not the full catalog)
openai	Any OpenAI-compatible (LocalAI, …)	8000	/v1/models
comfyui	ComfyUI (image / audio / video)	8188	/object_info/CheckpointLoaderSimple — SD checkpoints

---

Architecture

A three-crate Rust workspace:

• network/ — the libp2p swarm: Kademlia DHT, mDNS, Circuit Relay v2, DCUtR, AutoNAT v2, UPnP, and the request/response proxy. Exposes a synchronous NetworkHandle to the agent and the openhydra-bootstrap binary.
• protocol/ — pure protocol logic: canonical model-id resolution, capability records, the router (resolve → rank → route), co-signed receipts, and the redb receipt ledger.
• agent/ — the runnable host (openhydra-agent): engine adapters, the provider loop, and the OpenAI-compatible consumer gateway.

Public bootstrap nodes (a small Rust openhydra-bootstrap binary on a few VPS instances, dual-stack IPv4+IPv6 on :4001) seed DHT discovery and act as relays for NAT'd peers. Deploy scripts live in ops/bootstrap/.

---

Benchmarks

Historical throughput data from earlier builds lives in benchmarks/. Because OpenHydra now proxies to an external engine, end-to-end throughput is the engine's own tokens/sec plus the transport overhead (LAN direct ≈ a few ms; relayed cross-NAT paths add a hop).

---

Security & Privacy

• Each node's ed25519 identity key never leaves the Rust daemon. Receipts are co-signed in-process; only the resulting signatures and public keys cross the wire.
• Transport is authenticated and encrypted by libp2p (Noise + TLS over QUIC/TCP).
• Provider operators choose what their engine exposes; the protocol routes by model id and does not require sharing prompts with any third party beyond the serving provider.
• Token-count checks fetch the whole tokenizer index (it says nothing about which models you use), but downloading a model's tokenizer from Hugging Face tells Hugging Face that this gateway uses that model. Prompts and answers are never sent there; serve --no-tokenizer-downloads (or the desktop's "Check billed token counts" switch) turns the download off.

See SECURITY.md to report vulnerabilities.

---

License

Apache 2.0 — see LICENSE.