collegedata.fyi

College facts pulled from school-published Common Data Sets and source-labeled federal data.

Open-source Common Data Set API, searchable college admissions dataset, federal baseline fact layer, and preservation archive for U.S. higher education data.

An open, reproducible library of US college data. We find each school's Common Data Set document, extract it into a canonical schema, and publish both the raw source file and the structured extract alongside a queryable manifest. For schools without a public CDS, we now publish a curated NCES/IPEDS federal baseline keyed by ipeds_id (zero-padded UNITID) and labeled as federal data. No hand-cleaned numbers, no unlabeled source blending: CDS remains the school-authored source, IPEDS and Scorecard remain federal context.

Status: V1 live at collegedata.fyi. The August 6, 2026 integrity audit counted 6,322 institutions in the federal directory, 4,071 archived CDS documents, 262,537 public cds_fields rows, and 558 school_browser_rows. Five of six extraction tiers shipped: filled XLSX, fillable PDF, flattened PDF, image-only scan, and structured HTML. DOCX is the only remaining tier and is scoped in PRD 007. The live product now includes institution coverage transparency, source-labeled NCES/IPEDS baseline facts, academic positioning, admission strategy, match-list building, merit profile data, Scorecard outcomes, and a PRD 019 change-intelligence alpha for source-linked year-over-year CDS deltas. Public change events are explicitly review-gated. See the August 2026 data-integrity audit, docs/extraction-quality.md for the last extraction-accuracy benchmark, and tools/ipeds/README.md for the IPEDS release pipeline.

Why this exists

There is no free public API for Common Data Set information. Every school publishes to its own URL, most as PDFs, with no central index. If you want to compare admissions statistics across schools, the options today are "write a custom scraper for each institution," "pay a commercial data provider," or fall back to IPEDS. IPEDS is the right federal baseline for broad coverage, but it lacks the current-year, school-authored admissions granularity the CDS captures.

Two recent discoveries made this project much cheaper to build than it would have been a year ago:

1. The CDS Initiative publishes a canonical machine-readable schema in the official 2025-26 Excel template. We extract it programmatically — 1,105 fields keyed by stable question numbers — so there's no schema-design work, and every school's data lands in the same shape.
2. A meaningful minority of school CDS PDFs are actually unflattened fillable forms with named AcroForm fields. For those schools, extraction is deterministic via pypdf.get_fields() and matches ground truth perfectly. Harvey Mudd 2025-26 is the verified case.

Combine those two and an open CDS library that was a multi-month engineering project a year ago is now a weekend's worth of effort. That is the actual reason this project exists now.

We also archive source files on discovery, because some schools do occasionally remove historical CDS from their websites — MIT's 2023-era CDS URLs, for example, were all removed during a 2024-2026 domain migration. This is a side benefit of the architecture, not the headline.

What's here

• collegedata.fyi — a public frontend for browsing, searching, and downloading archived CDS documents, plus an institution directory of every active Title-IV school whether or not we have a CDS for it
• Source files (PDF, XLSX, DOCX, and structured HTML) for each school + year combination we've found, archived on discovery
• Canonical structured extracts keyed to the CDS Initiative's own field IDs (A.001, B.101, C.101, ...), with provenance linking every value back to the source
• An institution directory and coverage transparency layer (PRD 015) — every active, undergraduate-serving Title-IV institution gets a searchable identity page and an honest CDS coverage status (CDS available / Older CDS available / No public CDS found / Not checked yet). Curated identities are checked against an official NCES snapshot before they can be written, and retired school URLs permanently redirect to the canonical identity. The public coverage table at /coverage makes the gap visible.
• A source-labeled NCES/IPEDS federal baseline (PRD 021) — curated enrollment, admissions, cost, aid, and outcome-adjacent facts for in-scope institutions, especially schools where no public CDS is archived. Provisional/final release status, source table/variable, imputation status, and definition-alignment notes stay visible.
• Fit-data products on top of the raw CDS — academic positioning (PRD 016), admission strategy (PRD 016B), match-list building (PRD 017), and merit profile data (PRD 018)

Quick look

Browse the site: collegedata.fyi — search for a school, view archived CDS years, download archived source files or a per-school-year XLSX/CSV spreadsheet of the extracted values, or browse extracted field values.

Query the API:

Simple no-auth endpoints for agents, CLIs, and notebooks:

curl 'https://www.collegedata.fyi/api/schools/search?q=mit'
curl 'https://www.collegedata.fyi/api/schools/mit/facts?categories=admissions,cost,outcomes'
curl 'https://www.collegedata.fyi/api/compare?schools=mit,yale,university-of-chicago'
curl 'https://www.collegedata.fyi/openapi.json'

Raw PostgREST remains available for power users:

ANON_KEY="<copy the public anon key from https://www.collegedata.fyi/api>"

# List all live schools in the manifest
curl 'https://api.collegedata.fyi/rest/v1/cds_manifest?removed_at=is.null&select=school_id,school_name,canonical_year&order=school_name' \
  -H "apikey: $ANON_KEY" \
  -H "Authorization: Bearer $ANON_KEY"

# Find a specific school's documents
curl 'https://api.collegedata.fyi/rest/v1/cds_manifest?school_id=eq.yale&removed_at=is.null&order=canonical_year.desc' \
  -H "apikey: $ANON_KEY" \
  -H "Authorization: Bearer $ANON_KEY"

# Get the structured extract for a document
curl 'https://api.collegedata.fyi/rest/v1/cds_artifacts?document_id=eq.<uuid>&kind=eq.canonical' \
  -H "apikey: $ANON_KEY" \
  -H "Authorization: Bearer $ANON_KEY"

# Historical IPEDS facts: use ipeds_id, field_key, and a data_year range for fast reads
curl 'https://api.collegedata.fyi/rest/v1/ipeds_facts?ipeds_id=eq.110635&field_key=in.(retention_rate_full_time,graduation_rate_6yr)&data_year=gte.2019&data_year=lte.2024&select=ipeds_id,data_year,field_key,value_numeric,source_table,source_variable&order=data_year.asc' \
  -H "apikey: $ANON_KEY" \
  -H "Authorization: Bearer $ANON_KEY"

How it works

1. Supabase Edge Functions run on cron, discover new or changed CDS documents at each school's Institutional Research URL, and record them in Postgres. Source bytes are archived in Storage on first discovery so we still have them if the school later removes the original. Schools that publish one CDS year as separate A-J section PDFs are assembled into one logical extraction source while preserving each original section file as provenance.
2. A Python worker prioritizes fresh CDS years, routes each document from byte-derived source_format, and extracts by tier. Tiers that ship today: filled XLSX -> template or embedded answer-column cell map + openpyxl; fillable PDF with AcroForm fields -> deterministic direct read (tools/tier2_extractor/); flattened PDF -> Docling layout extraction + schema-targeting cleaner (tools/extraction_worker/tier4_cleaner.py); image-only scans -> force-OCR pass through the same Docling pipeline; structured HTML -> HTML normalizer reusing the Tier 4 cleaner. Remaining tier scoped but not yet built: filled DOCX via Structured Document Tags (PRD 007).
3. All extractors produce output keyed to CDS canonical field IDs using the schemas at schemas/. 2025-26 and 2024-25 are canonical template years; 2023-24 is supported by a synthesized canonical schema built from the official 2023-24 PDF form tags plus the 2024-25 field map. Older structural schemas remain available for audits and overlays.
4. The IPEDS pipeline loads official NCES metadata workbooks and selected CSV table ZIPs into release, metadata, raw-row, and curated-fact tables. Historical releases from 2004-05 through 2024-25 can be loaded with Access fallback for older releases. school_facts_unified is the public serving view for source-labeled federal baseline facts; ipeds_current_facts is backed by a materialized cache; raw IPEDS rows are preserved for audit, not treated as the product API.
5. An operator-run FSA pipeline loads the public institutional nonpayment workbook into fsa_releases and fsa_nonpayment_facts, stamps school_id at load, and serves fsa_nonpayment_current. School pages cite FSA on the number. This is not the official cohort default rate and is not a fourth named homepage source.

Full architecture: docs/ARCHITECTURE.md.

Ecosystem and related projects

collegedata.fyi sits between official higher-education data systems and the document-processing tools needed to make decentralized CDS files queryable:

• Common Data Set Initiative — the canonical CDS templates and field definitions this project follows.
• College Scorecard API — federal outcomes, net-price, debt, completion, and earnings data; joined into cds_scorecard.
• IPEDS — federal postsecondary reporting system, source of UNITID identity metadata, and source for the PRD 021 federal baseline fact layer.
• Federal Student Aid Data Center — public institutional nonpayment workbook (nonpayment-rates.xlsx); not the official cohort default rate.
• Docling — open-source document conversion toolkit used for flattened PDF, scanned PDF, and layout-aware extraction.
• UrbanInstitute/ipeds-scraper — downloader for IPEDS complete data files.
• UrbanInstitute/education-data-package-r — R package for accessing education data including IPEDS and College Scorecard data.
• karllhughes/colleges — open API of U.S. colleges and universities.
• kielni/ipeds-sql — IPEDS data loaded into a SQL-friendly shape.

Docs and decisions

• docs/ARCHITECTURE.md — twelve-pipeline map of the whole system (schema, corpus, discovery, mirror, extraction, scorecard, institution directory + coverage, IPEDS federal baseline, FSA institutional nonpayment, change intelligence, consumer API, frontend)
• docs/ops-discord.md — Bolewood Discord #alerts / #deploys routing and the DISCORD_ALERTS_WEBHOOK_URL / DISCORD_DEPLOYS_WEBHOOK_URL GitHub secrets
• docs/data-extraction-pipeline.md — operational diagram of the discovery/archive/extraction/projection flow, including cadence, storage, and known issues
• docs/extraction-quality.md — current accuracy by tier, per-section corpus-wide coverage, and reproducible scoring commands
• docs/api-usage-attribution.md — low-PII friendly API usage attribution for MCP, CLI, and cooperative external integrations
• docs/recipes/ — worked examples with real data: interactive visualizations, XLSX starters, and API queries. Start with College Pricing Power
• docs/plans/prd-019-spike-and-qa.md — PRD 019 spike and QA summary, including the first calibration-run numbers and review gates
• — reproducible 24-check data-integrity audit, evidence manifest, findings, limitations, and recommended rerun cadence

Contributing

This project is designed around community contribution from day one. Two especially valuable ways to help right now:

Add a school to the discovery scraper. If collegedata.fyi doesn't know about a school's CDS, it's usually because the school's website doesn't match our default URL patterns. PR a new entry to schools.yaml with a known-good URL or search pattern.

Write a cleanup tool. Our in-tree cleaner handles the high-value 2024-25/2025-26 browser and school-page surfaces well, including admissions, SAT/ACT, Section H aid, and several large enrollment grids, but Docling markdown still has a long tail of format variants we don't cover (see docs/known-issues/, docs/extraction-quality.md, and the corpus survey). Raw Docling markdown lives at cds_artifacts.notes.markdown on every Tier 4 artifact, keyed by document. A cleanup tool reads the markdown, normalizes its target fields, and publishes the result as a new artifact with its own producer tag. Register your tool in cleaners.yaml and the CI will run it against the corpus. Per ADR 0002, we don't pick a winner — every contributor's artifact is published alongside the primary ones.

See CONTRIBUTING.md for the full contributor guide, including how to add a school, fix your own school's data, write a cleaner, and become a co-maintainer.

Authors

Created and maintained by Anthony S. at Bolewood Group, LLC. Contributions welcome — see the Contributing section above.

License

MIT — see LICENSE. Copyright © 2026 Anthony S. and Bolewood Group, LLC.

One exception: the curated discovery content under data/discovery/ (experience-card library, interest ontology, explanation templates, discovery policy definitions) is licensed CC BY-SA 4.0 — publicly inspectable, attribution and share-alike required. Contributions to that directory require a contribution agreement; see data/discovery/README.md.