Fivefold: a Big Five personality test that shows its work
A short Big Five (OCEAN) personality test that runs entirely in your browser. You answer 20, 30 or all 50 public-domain IPIP statements. For each of the five traits you get a percentile against hundreds of thousands of real respondents, an honest error band, and the answers that moved each score.
Live: https://personality-trait-predictor.vercel.app
Built with Next.js (static export), TypeScript and Tailwind. The model is trained in Python (ml/) and shipped as a JSON file of about 200 KB (about 65 KB gzipped).
How it works
Data
The model is trained on the Open Psychometrics IPIP-FFM dataset: about 1M anonymous online responses (2016-2018) to the 50-item IPIP Big Five markers. Cleaning steps:
| Step | Removed | Remaining |
|---|
| Raw | | 1,015,341 |
One submission per IP (IPC == 1) | 318,496 | 696,845 |
| No missing answers | 93,523 | 603,322 |
| Not a straight-liner (all 50 identical) | 731 | 602,591 |
| Completion time 50 s to 1 h | 5,491 | 597,100 |
The clean rows are split at random: 477,680 for training and 119,420 held out for testing.
Scoring (ground truth)
Standard IPIP scoring. Each trait is the mean of its 10 keyed items on a 1-5 scale, and a reverse-keyed item counts as 6 - x. Traits are reported in this order: Extraversion, Neuroticism, Agreeableness, Conscientiousness, Openness.
The short-form model
The model predicts your full 50-item scores from fewer answers.
- Question order. Greedy forward selection on the training split. Each step adds the statement that most raises the mean R² across the five traits. Under a linear-Gaussian model the remaining uncertainty after a set of questions doesn't depend on how they were answered, so asking adaptively gains nothing over a fixed order. The order itself is chosen greedily: a good approximation, not a guaranteed optimum (a one-swap search finds item sets about 0.005 R² better at k = 10-20, but those are not nested prefixes of one order).
- One linear model per prefix length k = 1..49 (OLS on raw 1-5 answers). At k = 50 no model is involved: the app uses the exact scoring formula.
- Uncertainty. Each k and trait stores the held-out residual SD (RMSE), shown as the "typical error". The band drawn around each result is conditional on the prediction: the training rows are cut into 12 equal-count bins of predicted percentile, and each bin stores the 16th and 84th percentiles of the residual (
stages[k-1].traits[t].band). A single pooled ±1 RMSE band was the wrong width near the ends of the scale: scores cannot exceed 5, and on skewed traits a fixed score band spans far more percentiles at the top than at the bottom. On the test split the conditional band covers 68% of people for every trait and length (target 68%).
- Wording. A result reads "Strongly" when the point percentile is beyond 85 (15) and the band's cautious end beyond 80 (20); "Clearly" needs 70 / 65 (30 / 35); otherwise "Leaning". With the old pooled band and a whole-band-beyond-85 rule, only 3% of people truly in the top 5% for Openness were told "Strongly high" at 20 statements, against 88% of the bottom 5% told "Strongly low". Now, at 20 statements, the two shares are within 13 points for every trait (worst: Agreeableness, 85% vs 98%), and people told "Strongly" are on that side of the 70th / 30th percentile at least 97% of the time. Below 20 statements high Agreeableness and Openness are still worded more cautiously than low (at 15 statements, 4% vs 89% for Agreeableness): the full-scale scores bunch up below the ceiling, so the short form genuinely separates high scorers less well.
ml/reports/evaluation.md → "Result wording" has the table.
- Shown percentile. It is the percentile of the predicted score, which a short form pulls toward the average, so very high Openness and Agreeableness percentiles read a few points low at short lengths (at 10 statements nobody is shown above about the 93rd for either).
- Response validity. Straight-liners were removed from the training data, so the results page flags answer patterns outside it: every answer identical, an answer SD below the 0.1% training quantile for that length, or a mean answer outside the 0.5-99.5% training range (agreeing or disagreeing with nearly everything). The thresholds ship in
model.json → . Results also need at least 10 answers; shorter links ask the person to continue the test.
Percentiles
Each trait is reported on its own scale, as a percentile against the training split's full-scale scores (1,001 stored quantiles, linear interpolation). The trait shown as "most distinctive" is the one furthest from the 50th percentile, not the one with the largest raw number.
Evaluation (held-out test split, 119,420 people)
Mean R² across the five traits, predicting the full 50-item score:
| Model | k=10 | k=20 | k=25 | k=30 | k=40 |
|---|
| Linear (shipped) | 0.754 | 0.883 | 0.917 | 0.942 | 0.978 |
| LightGBM | 0.758 | 0.886 | 0.919 | 0.943 | 0.977 |
| Small MLP | 0.757 | 0.885 | 0.918 | 0.942 | 0.978 |
| Prorated baseline (score only answered items) | 0.506 | 0.800 | 0.863 | 0.895 | 0.967 |
- The nonlinear models gain at most about 0.002 R² at k = 25, well under the 0.01 bar set in advance, so the linear model ships. It is tiny, exact to explain, and runs client-side.
- Typical error (mean RMSE on the 1-5 scale): 0.26 at 20 questions (Quick), 0.19 at 30 (Standard), 0 at 50 (Full, exact scoring).
- Bias check. At k = 25, the trait furthest from the median (the one the results card names) is each trait for 19.1-20.8% of test respondents, close to an even 20%. The highest-percentile trait is less even (Neuroticism 25.4%) because of how the traits correlate: Neuroticism runs opposite to Extraversion (r ≈ −0.22) and Conscientiousness (r ≈ −0.24) and is barely related to the other two, so when the others are low it tends to be the maximum, while Agreeableness correlates positively with Extraversion (r ≈ 0.31) and splits the wins with it. Exact 50-item scoring gives the same 25%, so this is trait correlation, not model bias.
- Does the short form pick the same trait as the full test? For the most-distinctive trait shown on the results card: 55% at 10 questions, 68% at 20 (Quick), 73% at 25, 77% at 30 (Standard), 86% at 40, 100% at 50 (Full). For the highest-percentile trait at k = 25 it is 80.4%. Chance is 20%.
The full report, with per-trait R², RMSE, MAE and Pearson r, the confusion matrix and plots, is in ml/reports/evaluation.md. The same numbers appear on the site's /method page, generated from the shipped model.json.
Why v1 always gave the same answer
The first version (a Streamlit app with five CatBoost models) told nearly everyone they had the same "dominant" trait. The original bug report named Agreeableness. A faithful replay of the committed code on 5,000 held-out respondents gave Openness for 88.9% and Agreeableness for 9.5%. Its top trait matched the person's real top trait only 12% of the time, worse than chance (20%). Several bugs combined to cause this:
- Scale mismatch. The scaler and imputation means were fit on centred answers, but the app fed in raw 1-5 answers. Every input landed far outside the training range. Answering all 2s, all 3s, all 4s or all 5s produced identical outputs.
- A softmax across traits. The five nearly constant outputs went through a temperature-0.15 softmax, so whichever trait saturated highest won almost every time.
- Wrong direction. A textbook extravert scored lower on Extraversion than a textbook introvert (0.64 vs 0.75), and lower than someone answering neutral to everything (0.97).
- The wrong question. Big Five traits are separate dimensions, each on its own scale. They are only modestly correlated (in this data, for example, Extraversion and Agreeableness r ≈ 0.31), and being high on one does not make you lower on another, so a percentage "share" split across them has no meaning.
v2 replaces all of this: correct scoring, one linear model per test length, a separate percentile for each trait, and the bias check above.
Local development
Requires Node 20.9+ (developed on Node 24).
npm install
npm run dev # http://localhost:3000
npm run build # static export to out/
npm start # serve out/ locally
| Command | What it does |
|---|
npm run lint | ESLint |
npm run typecheck | tsc --noEmit |
npm test | Vitest unit tests: inference, parity with Python, share links, export |
npm run test:e2e | Playwright end-to-end tests against the static build (desktop 1440 px and mobile 375 px) |
npm run sync-model | Validate ml/artifacts/model.json, copy it to src/data/model.json and regenerate src/data/model.runtime.json (also regenerated before every build) |
npm run og-image | Regenerate the Open Graph image and touch icon |
Layout:
src/lib/model.ts pure inference (predict, percentiles, contributions)
src/data/model.json the trained model (copied from ml/artifacts)
src/data/model.runtime.json slim slice of it for /test (order, statements, per-k error); generated
src/data/content.ts trait descriptions and citations
src/app/ /, /test, /results, /method
ml/ training pipeline, tests and reports (not deployed)
Retraining (ml/)
python -m venv .venv && . .venv/bin/activate
pip install -r ml/requirements.txt
python ml/pipeline.py --data path/to/data-final.csv --out ml/artifacts --reports ml/reports --seed 42
npm run sync-model
If --data doesn't exist, the pipeline downloads the dataset. The dataset is never committed. A full run takes about 7-8 minutes on a laptop and is deterministic for a given seed. Use --sample 50000 for a quick run. The pipeline writes ml/artifacts/model.json, ml/artifacts/parity_cases.json (30 reference predictions, with their bands and wording, that the TypeScript code must reproduce) and ml/reports/evaluation.md.
model.json follows the contract in SPEC.md, plus optional extras the web app reads when present: per-stage conditional bands, responseStyle, and in evaluation the per-length byK and strength tables, traitCorrelation and shrinkage.
Tests
python -m pytest -q ml/tests # scoring, cleaning, model contract, sanity profiles, parity cases
PTP_DATA=path/to/data-final.csv python -m pytest -q ml/tests/test_heldout.py # re-checks wording and band coverage on the held-out split
npx vitest run # TS inference + parity with Python (scores within 1e-6, percentiles within 1e-4)
npx playwright test # full test flow, results, share links, /method, no horizontal scroll at 375 px
Privacy
Everything runs in your browser. Answers are never sent anywhere. There is no server, no analytics and no cookies. A share link carries your answers in the URL fragment (#…), which browsers do not send to the server.
Credits
This is a self-report personality description, not a clinical or hiring tool.
