[!WARNING]
V1 is a release candidate. Install viteval@rc. Untagged installs resolve to v0.
V0 support ends when v1 launches. The v0 branch contains the stable release.
Features
Viteval is an independent open-source project. It is not affiliated with,
endorsed by, or supported by VoidZero.
- Viteval-owned configuration and CLI, with Vitest-powered execution, watch mode, retries, and CI exit codes.
- Typed tasks, reusable datasets, built-in scorers, and custom scoring functions.
- JSON storage by default, with atomic writes. Inspect saved results locally with
viteval ui.
- Optional SQLite, Braintrust, Langfuse, and hosted VoltOps providers for datasets and evaluation results, plus S3/GCS dataset storage. Backend SDKs and database drivers are opt-in.
- CLI commands for project setup, dataset generation, and the dashboard.
Installation
pnpm add -D viteval@rc
pnpm exec viteval init --no-env-file
pnpm exec viteval run
pnpm exec viteval ui
Requires Node >=26. Initialization creates viteval.config.ts (.mts for
CommonJS) and a passing src/viteval.eval.ts example. JSON results are saved
under .viteval/results. Existing Viteval configuration is preserved; no runner
config, setup module, or reporter is required.
Usage
Define an evaluation in src/uppercase.eval.ts. This example runs without a model or API key:
import { evaluate, scorers } from 'viteval';
evaluate('Uppercase', {
data: [
{ input: 'hello', expected: 'HELLO' },
{ input: 'world', expected: 'WORLD' },
],
task: ({ input }) => input.toUpperCase(),
scorers: [scorers.exactMatch()],
});
Run it locally or in CI:
pnpm exec viteval run
Replace task with your model, agent, or workflow. Each case passes when its
configured mean, median, or sum meets the threshold. Failed execution and scorer
errors fail the case and remain inspectable in the JSON report.
Development
pnpm validate
pnpm validate:consumer
pnpm --filter @viteval/cli test:binary
pnpm docs:types
pnpm docs:build
The consumer check installs packed packages outside the workspace and verifies
initialization, evaluations, JSON datasets, SQLite, server-rendered dashboard
deep links, JavaScript/CSS delivery, and API/404 behavior. Run
pnpm dev:ui examples/basic to develop the dashboard with Vite on port 6274.
See v1 completion for the release contract and evidence.
