Shared, local infrastructure for coding agents. A single NVIDIA GB10 system
(DGX Spark) runs vLLM with a coding model, and
concurrent users run OpenCode on their own workstations
against it. It works the same for a classroom or a company team: prompts and source code
stay on the organization's network.
The repository contains scripts, example configuration and documentation to
install the service, secure it, test it, measure it and operate it.
Status: 0.2.0. CI tests every script and configuration file
offline. The service has also run on one GB10 with a simulated class of
concurrent OpenCode users (see Benchmarks). Treat those figures
as a reference for this model and client version, not as a promise of
performance.
A web panel for registration, keys, OpenCode setup and a live
dashboard is included as a preview.
How it works
- The GB10 only serves inference. Users' code, version control, compilers,
interpreters, package managers, builds and tests run on the workstations or in
dedicated isolated development environments, whatever the language or
platform. Do not run users' builds on the model host: CPU and GPU share the
same memory.
- Every service runs in Docker. vLLM listens on the loopback interface only,
and an nginx container is the only way in. nginx accepts HTTPS on a dedicated
port from the organization's LAN or VPN, checks a per-user API key and
forwards only
/v1/chat/completions and /v1/models. The scripts in this
repository run on the host: they drive Docker and act as test clients, and
they install nothing on the system.
- OpenCode uses an explicit OpenAI-compatible provider. The served model name
is
coder, tool calling is enabled explicitly, and the context limit matches
the server profile.
Web panel (preview)
The web panel handles the class from the browser. People
register, get their key and connect OpenCode with one command. Administrators
approve people and follow vLLM, the GPU and the users live.
Screenshots from the test setup, with example accounts, a mock vLLM and a
simulated GPU: the figures are not measurements.
Quick start
On the GB10 host, with Docker and the NVIDIA runtime that DGX OS provides:
git clone https://github.com/stefanoferi/codendum.git
cd codendum
scripts/preflight.sh
cp .env.example .env
scripts/start-vllm.sh --wait
scripts/start-proxy.sh --init
scripts/start-proxy.sh
scripts/smoke-test.sh
Before start-proxy.sh, review .env and install three things:
- the host name and allowed networks in the proxy configuration;
- a TLS certificate;
- per-user API keys from
scripts/gen-api-keys.sh.
The installation guide and the
proxy guide explain every step. Then configure OpenCode on each
workstation from the server's own limits:
scripts/configure-opencode.sh --base-url https://llm.lab.example:8443
Documentation
The project website is https://codendum.ai, with a link to the published documentation.
Its sources in docs/ are Markdown files built with
Sphinx and
MyST.
| Page | Contents |
|---|
| Sizing | Unified memory, KV cache per token, connected vs active vs waiting users |
| Installation | Prerequisites, configuration, launching and upgrading vLLM |
| HTTPS proxy | nginx container, API keys, limits, queueing and fairness |
| OpenCode | Provider configuration and a real read-edit-test check |
| Smoke test | Health, streaming and tool-call checks |
| Benchmarks | Load tests, the classroom simulation and results measured on a GB10 |
| Profiles | Serving profiles and how to switch them |
| Troubleshooting | Common failures, logs and what to do |
| Security and limits | Threat model and responsibilities |
| Governance | What is governed today, and a possible future integration with Admina |
To build the HTML site locally:
python3 -m venv .venv
.venv/bin/pip install --require-hashes -r docs/requirements.txt
.venv/bin/sphinx-build -W --keep-going -b html docs docs/_build/html
Roadmap
A possible next step is governance of the content that goes through the model:
personal-data redaction, a prompt-injection firewall, agent loop breaking and a
tamper-evident audit log. It could be provided by integrating
Admina, an open-source framework for governed AI, as a
gateway between the proxy and vLLM. See Governance. The
integration is not implemented yet.
Contributing and security
Contributions are welcome: see CONTRIBUTING.md and the
Code of Conduct. Report vulnerabilities privately as
described in SECURITY.md.
License
Codendum is licensed under the Apache License 2.0; see also
NOTICE. Container images, model weights and client software are
downloaded separately and keep their own licenses: see
Reference.
Created and maintained by Stefano Noferi (GitHub:
stefanoferi).
