HumanStudy-Hub Web
Private Next.js application for HumanStudy-Hub.
Product areas
- Dataset catalog for the initial test suite and community studies
- Human-study builder with PDF/OSF intake and human review
- Playground for running a model through a study and comparing it to the people
- HumanStudy-Bench agent evaluations and leaderboard
- Research projects, partners, and sponsorship
Local development
npm install
cp .env.example .env.local
npm run dev
Open http://localhost:3000.
The account-based Build Study workspace is at /build; /build/example is an offline example in the same UI, and /build-preview redirects there. See workspace setup
for the Supabase migration, private PDF storage, original agent integration, and verification
steps. The implementation plan is in docs/build-study-platform-plan.md.
Legacy Build Study jobs at /pipeline are stored in the private HumanStudy-Hub/humanstudy-hub-jobs
repository. The Vercel API dispatches the HumanStudy-Bench GitHub Actions
workflow; Claude Code, document tools, and package validation run on the Actions
runner, not inside Vercel.
Verification
npm run lint
npm run test:studio
npm run build
Releases
Target feature pull requests to dev. Verify the integrated changes with CI and the Vercel preview.
For production, open a pull request from dev to main and wait for explicit approval from the release owner before merging.
Shared Build Study backend (/pipeline and /build)
The browser uploads the paper directly to Vercel Blob and posts only the
resulting URL to the API, because Vercel rejects any function request body over
4.5 MB and most journal PDFs are larger than that. Create a Blob store in the
Vercel project and set BLOB_READ_WRITE_TOKEN; Build Study cannot accept
uploads without it. The store uses private access, so the Actions runner
downloads the paper through a signed link recorded in job.json and commits it
to the job branch.
Blob is staging rather than storage: once the runner commits the paper, the
branch holds the durable copy. A daily cron calls /api/blob/cleanup to delete
anything older than two days, which also collects uploads that never became
jobs. Set CRON_SECRET in the Vercel project — the route refuses to run without
it, since it would otherwise let anyone empty the store.
The Node server creates a private job branch and starts a Claude Code agent
through GitHub Actions. The agent uses the paper and optional open-material URL
to build and validate the complete package, then pauses once for final
researcher review. Approval enables ZIP download and optional benchmark
contribution. Set GITHUB_TOKEN to enable job storage, workflow dispatch, and
publishing the final package as a contribution branch and pull request.
The legacy pipeline never calls OpenRouter from Vercel. Store OPENROUTER_API_KEY as an
Actions secret in HumanStudy-Hub/HumanStudy-Bench; the one Vercel variable of
the same name is used by the playground's persona designer and the new
interactive studio chat. The workflow also expects
the HUMANSTUDY_PIPELINE_TOKEN Actions secret and accepts an optional
OPENROUTER_MODEL Actions variable, which defaults to moonshotai/kimi-k3.
Playground backend
/playground dispatches run-playground.yml in HumanStudy-Bench the same way
Build Study dispatches its workflow. Run state, progress, and results live on a
runs/<id> branch of the private jobs repository, and the page polls
/api/playground/runs/<id>. Nothing about a run executes inside Vercel.
The runner replays the study with the chosen model through OpenRouter, scores it
with the study's own evaluator, and then has Claude Code chart and interpret the
result. Charts are validated on the runner before they are stored, and a run
that produces no usable chart set falls back to deterministic charts.
Runs use the OPENROUTER_API_KEY Actions secret with a small participant cap. A
researcher can paste their own OpenRouter key to run at full size; the key is
sealed with AES-256-GCM before it leaves the server, so the jobs repository only
holds ciphertext and the workflow drops it when the run ends. Set
PLAYGROUND_KEY_SECRET here and as an Actions secret in HumanStudy-Bench with
the same value — without it, the playground still runs on the shared key and
only refuses researcher-supplied keys.
Persona groups
A persona group describes who the agents in a run are. It is a population
rather than a fixed cast: each segment carries a share of the participants and
the ranges its members are drawn from, so one saved group fits a run of any size
and any study. lib/persona-groups.ts mirrors playground/personas.py in
HumanStudy-Bench, which does the sampling — the two validators and the
largest-remainder split must stay in step, so the mix a researcher previews is
the mix the run produces.
The designer chat calls OpenRouter directly from the server and requires
OPENROUTER_API_KEY here. It is the one part of the product that does, because
a design conversation needs replies in seconds; everything that touches a study
still runs in Actions. The model returns a whole group as JSON, which is
validated before it reaches the editor, and every field stays editable by hand.
Groups can be downloaded as JSON, loaded back, or contributed to the benchmark,
which opens a pull request adding
playground/profiles/<study>-<contributor>-<n>.json with the number rising each
time the same contributor saves another.
The production server only needs GitHub API access. Do not add PDF files,
pipeline outputs, or tokens to the public web repository.
