REAL Fighter Rankings
Results, Evidence, Analytics, Ledger. Open, reproducible UFC rankings built from public data: a weekly resume board (who has earned it), a separately graded prediction model (who would be favored), and a card-quality index for matchmaking research. Companion to the SSAC27 abstract in paper/abstract_ssac27.md.
- Site: https://braytonsmith-dev.github.io/real-fighter-rankings/ (boards, side-by-side with public boards, prediction model, methodology)
- Methodology, version 1.0, with every rule, weight, and worked example: METHODOLOGY.md (regenerated on every rebuild from
config/weights.yaml and the audit trail)
- Per-fighter audit trail:
outputs/audit_top30.csv; forward validation: outputs/forward_validation.json
Principle: a fighter is ranked on how he performed against the fighters he faced, and how good those fighters were.
What it produces
| Output | File |
|---|
| Divisional boards, champion plus top 30, movement since the last event | outputs/composite_boards.md, docs/index.html |
| Audit trail: score order, rule moves, ledger detail for every contender | outputs/audit_top30.csv |
| Weekly snapshots and forward validation | outputs/history/, outputs/forward_validation.json |
| Prediction model ratings and backtest | docs/prediction.html, outputs/backtest_report.json, outputs/table_backtest.md |
| Comparison with UFC media panel, Meta, Sherdog, Fight Matrix, ESPN | outputs/compare_report.md, outputs/compare_flags.csv |
| Card-quality index and supply facts | outputs/card_quality_*.csv, outputs/card_quality_supply.json |
| Fighter profiles: why each board fighter is ranked there, last ten bouts, career statistics, trajectory | docs/fighters/<slug>.html, docs/fighters/index.html |
| Decline and rise index: the rating's age curve and each fighter's own level and trend, display only | outputs/decline_index.csv, outputs/decline_curve.json |
Headline results (as of 2026-09-26, method v1.0 frozen 2026-10-01)
Prediction, held-out 2020-2026 (3,390 bouts; parameters fixed by a 520-point grid on 2010-2019 and scored once):
| Model | Log loss | Brier | Accuracy |
|---|
| Coin flip | 0.693 | 0.250 | 50.0% |
| Results-only Elo, tuned | 0.674 | 0.241 | 58.1% |
| Performance-adjusted Elo (in-fight dominance) | 0.663 | 0.235 | 60.6% |
The log-loss gain of 0.011 has a 95% event-block bootstrap interval of 0.006 to 0.016 (McNemar exact p = 0.002 on the picks). On the 1,445 held-out bouts with closing odds (2020-2023) the de-vigged market scored 0.609 log loss and 67.1% accuracy against the model's 0.669 and 60.0%: the model is a transparent rating, not a betting edge. The first-listed (red corner) fighter wins 57.2% of held-out bouts against 53.5% predicted; the model carries no corner term. Full table: outputs/table_backtest.md; per-bout ledger: data/processed/bouts_rated.csv.
Card quality, 203 cards from 2022 to Sept 2026: numbered events average 2.19 champion-or-top-10 bouts per card (1.26 excluding title bouts) against 0.72 (0.71) for Fight Nights; 44% of Fight Nights carry none. Fighters starting a year inside a division top 11 average 1.30 to 1.44 bouts that year and 12% to 17% do not fight at all.
Resume board (REAL v1.0): a retrospective reconstruction over 127 boards is reported in METHODOLOGY.md section 8 and is not a forward test. The prospective test is pre-registered in PREREGISTRATION.md and starts with the first event after Oct 1, 2026. Data rights and redistribution basis: DATA_LICENSE.md.
How this was built
The author designed the ranking principles, made every modelling decision recorded in METHODOLOGY.md and PREREGISTRATION.md, and reviewed all outputs. The code, documentation and analysis were written with the help of an AI assistant (Anthropic's Claude) working under the author's direction; commits carry a Co-Authored-By trailer to record that. Mistakes are the author's responsibility.
Pre-event picks
Every Monday and Friday the pipeline reads the upcoming UFC schedule, writes a pick for each bout inside the next three weeks to predictions/picks.csv (frozen performance-adjusted Elo probability, pick, confidence band, provisional flag, UTC timestamp) and never edits it. After each card the picks are graded from UFCStats and the running record (accuracy with interval, log loss, calibration by confidence band, baselines on the same bouts) goes to outputs/picks_summary.json and docs/picks.html. Each row also carries the pick of shadow v1.1 (src/mmalab/shadow.py: Elo re-tuned walk-forward over 2011 onward with a red-corner term, plus a logistic layer on age, layoff, experience, height and five-round bouts; development results in outputs/shadow_v11_report.json), which is graded alongside v1.0 and adopted only under PREREGISTRATION.md appendix B. Rules: PREREGISTRATION.md, appendices A and B. Settings: config/picks.yaml; publish_page: true adds the page to the site navigation.
Market baseline (off by default): src/mmalab/odds.py can record the de-vigged bookmaker consensus for every pending pick, once at lock time and once on the last run before the event, to predictions/market_at_lock.csv, so each miss can be read against the market on the same bout. It needs a free key from The Odds API in the GitHub Actions secret ODDS_API_KEY and enabled: true in config/odds.yaml; only derived probabilities and the number of books are stored publicly. Setup, what is stored, and the terms with citations: MARKET_BASELINE.md.
Build it yourself, step by step
Measure twice, cut once. Each step has a check so you know it worked before moving on.
以上为 README 的开头部分。其余内容请阅读原文。 查看完整 README ↗
