IQ visual reasoning gym

A visual reasoning gym for humans and AI agents — the same puzzles, two audiences.

Items are generated from a fresh random seed after the test starts. Every item carries a machine-readable rule or constraint witness, and pure code re-derives its answer. Humans and vision models receive the same answer-free visual contract.

Humans get a score and a breakdown by reasoning family. Agent runs report performance by difficulty tier and model, making it possible to compare where people and models find the same generated questions easy or hard.

The public product offers a 5-question sample and a 30-question test drawn from the same generated visual-family pool. Answers stay encrypted until server-side submission. Design: docs/plans/deterministic-novel-tests.md.

Question order

Each question number has a fixed difficulty. The sample uses 2, 3, 4, 4, 5. The long test uses level 2 for questions 1–5, level 3 for 6–9, level 4 for 10–22, and level 5 for 23–30. Each slot randomly selects a family validated at its level. The long test covers every enabled family; the sample uses five distinct families. Consecutive questions use different families, and the reference fallback preserves the same difficulty curve. These levels are structural estimates. Human feedback and timed pilot attempts are needed to check that different families at the same level are comparably hard. Level-5 machine questions always use all three transforms in a mixed order, excluding the demonstrated order. Every demonstrated machine transform is necessary to choose among the six answers. Ignoring any one leaves at least two choices consistent with the other examples. Generation and fallback serving enforce this rule.

Test generation latency

Measured 2026-09-29 on the development machine on scene-families-v19, fresh random seeds:

Test	Seeds	Median	95th percentile	Slowest
30 questions	50	321 ms	434 ms	502 ms
5 questions	100	54 ms	129 ms	232 ms

The browser gives up on starting a test after 20 seconds, so this leaves a wide margin even on slower serverless hardware.

---

Quick start

npm install
npm run dev          # http://localhost:3000

Local scoring requires the Upstash settings described below. Production must set a stable QUIZ_TOKEN_SECRET of at least 32 characters for answer-key encryption. The agent harness also needs relay settings; copy .env.example → .env.local:

RELAY_BASE_URL=http://localhost:8787/v1   # the local relay (../llm-relay under wrangler dev)
RELAY_PROVIDER=codex                      # your own Codex sign-in, through the relay's harness bridge
RELAY_MODEL=default                       # whatever Codex defaults to; runs record the model that answered
RELAY_EFFORT=high

Without a local relay and bridge, use the hosted relay with a vision model instead, as the recorded probes did: RELAY_BASE_URL=https://llm-relay.ebarakos.workers.dev/v1, RELAY_PROVIDER=openrouter, RELAY_MODEL=google/gemini-3.5-flash, plus OPENROUTER_API_KEY.

No model call is made when a human starts or submits a test. Relay access is used only when an agent takes the visual test or during offline model experiments.

Vercel: one-time setup, then push

Run this once from a clone with an origin remote:

npm run vercel:setup

The command uses the official Vercel CLI through npx, links or creates the project, creates separate sensitive QUIZ_TOKEN_SECRET values for production and preview when they are missing, connects origin to Vercel, and assigns iq.ebarakos.com to the production project. Secret values go directly from memory to Vercel: they are never printed or written to disk. Existing values are kept, so rerunning setup does not invalidate an active test. The domain is never moved with --force; if another project owns it, setup stops instead of breaking that site. The first run may ask you to sign in to Vercel; after it completes, normal pushes trigger preview deployments and pushes to the production branch trigger production deployments.

The setup command prints Vercel's domain inspection after assignment. If DNS lives elsewhere, add the exact A or CNAME record reported there. The current Cloudflare setup requires an A record named iq pointing to 76.76.21.21, with Cloudflare's proxy disabled. Vercel provisions HTTPS after DNS verification.

Use npm run vercel:setup -- --dry-run to inspect the intended work without contacting Vercel.

Upstash Redis

Connect the Upstash resource to this Vercel project for Preview and Production. The integration supplies KV_REST_API_URL and KV_REST_API_TOKEN. For local verification, copy the REST URL and write token from Upstash into the ignored .env.local file alongside existing relay settings. Vercel cannot export variables marked Sensitive; a connection with sensitive variables cannot target Development.

npm run redis:verify

This checks authentication, write/read access, expiry and atomic duplicate protection, then removes its unique test key. The connection uses native fetch, with no new package. Server code can call redisCommand in src/lib/redis.ts. Both in-page and link-test submissions save its first result before revealing answers. Identical retries return that result; changed answers are refused. Redis failures leave the test retryable and reveal no answer key. These private receipts expire with the two-hour test token. New Vercel environment settings take effect on the next deployment.

This is the beginning of the README. Read the rest at the original. Read the full README ↗