QVAC

QVAC lets you run a comprehensive range of AI workloads locally using open models across Linux, macOS, Windows, Android, and iOS.

QVAC provides:

• SDK for building local-first AI applications and systems in JavaScript/TypeScript and Python.
• HTTP server for using QVAC as a local model provider. Its OpenAI-compatible API lets you connect AI tools such as OpenCode and OpenClaw, or any other compatible tool.

Why QVAC

• Local-first: run AI offline with inference optimized for commodity hardware, from consumer apps and embedded systems to on-premises deployments.
• Privacy and control: keep data local, own the AI system. No cloud or third-party APIs required.
• One SDK, all of AI: a comprehensive range of AI capabilities through one interface.
• Cross-platform: one codebase for Linux, macOS, Windows, Android, and iOS, using JavaScript/TypeScript or Python.
• Peer-to-peer: fetch AI models directly between peers and build unstoppable internet systems, like BitTorrent or IPFS, but for AI.

Quickstart

Load a model and run inference locally in a few steps. Pick your path.

Run your first example using the JS/TS SDK.

1. Create the examples workspace:

mkdir qvac-examples
cd qvac-examples
npm init -y && npm pkg set type=module

2. Install the SDK:

npm i @qvac/sdk

3. Create qvac.config.json to enable client and server logs during the run:

{
  "loggerConsoleOutput": true,
  "loggerLevel": "info"
}

4. Create the quickstart.js script:

import { loadModel, LLAMA_3_2_1B_INST_Q4_0, completion, unloadModel } from '@qvac/sdk';
try {
  const modelId = await loadModel({
    modelSrc: LLAMA_3_2_1B_INST_Q4_0,
    onProgress: (p) => {
      const mb = (n) => (n / 1e6).toFixed(1);
      const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`;
      process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`);
      if (p.percentage >= 100) process.stderr.write('\n');
    },
  });
  const history = [{ role: 'user', content: 'Explain quantum computing in one sentence' }];
  const result = completion({ modelId, history, stream: true });
  for await (const token of result.tokenStream) {
    process.stdout.write(token);
  }
  await unloadModel({ modelId });
} catch (error) {
  console.error('✖', error);
  process.exit(1);
}

5. Run the quickstart script:

QVAC_CONFIG_PATH=./qvac.config.json node quickstart.js

You'll see the model download first. Then QVAC will stream the response tokens and print them to the terminal.

Run your first example using the Python SDK.

1. Create the examples workspace:

mkdir qvac-examples-py
cd qvac-examples-py
python -m venv .venv
source .venv/bin/activate

2. Install the package (self-contained — bundles the QVAC worker and Bare runtime, no Node.js required):

# Replace <version> with the release you want, e.g. sdk-v0.17.0:
pip install tetherto-qvac-sdk \
  -f https://github.com/tetherto/qvac/releases/expanded_assets/sdk-v<version>

3. Create the quickstart.py script:

import asyncio
import sys

from tetherto.qvac_sdk import Client, completion, load_model, unload_model
from tetherto.qvac_sdk.models import LLAMA_3_2_1B_INST_Q4_0


def print_progress(p):
    line = f"▸ Downloading {p.percentage:.0f}% ({p.downloaded / 1e6:.1f}/{p.total / 1e6:.1f} MB)"
    print(line, end="\r" if sys.stderr.isatty() else "\n", file=sys.stderr)
    if p.percentage >= 100:
        print(file=sys.stderr)


async def main():
    async with Client() as client:
        t = client.transport
        try:
            model_id = await load_model(
                t, model_src=LLAMA_3_2_1B_INST_Q4_0, on_progress=print_progress
            )
            run = completion(
                t,
                model_id=model_id,
                history=[
                    {"role": "user", "content": "Explain quantum computing in one sentence"},
                ],
            )
            async for event in run.events:
                if event.type == "contentDelta":
                    sys.stdout.write(event.text)
                    sys.stdout.flush()
            print()
            await unload_model(t, model_id)
        except Exception as error:
            print(f"✖ {error}", file=sys.stderr)
            return 1
    return 0


if __name__ == "__main__":
    sys.exit(asyncio.run(main()))

4. Run the quickstart script:

python quickstart.py

You'll see the model download first. Then QVAC will stream the response tokens and print them to the terminal.

Launch the server with the CLI, then use QVAC as model provider for OpenAI-compatible tools like OpenCode and OpenClaw.

1. Install the CLI globally (this also installs @qvac/sdk as a transitive dependency):

npm install -g @qvac/cli

2. Create the examples workspace:

mkdir qvac-server
cd qvac-server

3. Create the qvac.config.json declaring one model to serve:

{
  "serve": {
    "models": {
      "my-llm": {
        "model": "QWEN3_600M_INST_Q4",
        "default": true,
        "config": { "ctx_size": 8192 }
      }
    }
  }
}

4. Start the server (bound to 127.0.0.1:11435 by default):

qvac serve openai

The model downloads on first start and is preloaded into memory. You'll see progress in the server output.

5. From another terminal, hit it with any OpenAI-compatible client. A minimal curl:

curl http://localhost:11435/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "my-llm",
    "messages": [{"role": "user", "content": "Explain quantum computing in one sentence"}]
  }'

The response comes back as a single JSON payload with the model's answer. Add "stream": true to the body to get an SSE stream instead.

6. Point your AI tool at the server: open its model provider settings and add a new OpenAI-compatible provider with base URL http://localhost:11435/v1, any string as the API key, and my-llm as the model name.

This is the beginning of the README. Read the rest at the original. Read the full README ↗