The web app now lives at verifycites.com/papers (the old szzxli.github.io/veripaper/web/ address redirects there).

veripaper — catch hallucinated citations in AI-written research drafts

Your AI wrote the draft. Did it invent the citations? veripaper checks every cited source against CrossRef, arXiv, Semantic Scholar and other registries, flags references that don't exist, and labels each claim honestly. Free and open source (MIT).

▶ Live demo: https://verifycites.com/papers/?ref=gh-veripaper — paste a draft or upload .docx/.pdf; everything runs in your browser, nothing is uploaded.

(veripaper is a working name.)

What it checks, per citation: does the cited work exist (DOI / arXiv ID / title lookup in free registries), does its metadata match (the authors you cited vs. the record), is it retracted, and — for a draft with claims — does the cited abstract support the claim. Input: a draft (markdown, plain text, .docx, .pdf) or a BibTeX .bib. It is a first-pass screen: use it to decide which references to check by hand, not as a substitute for human review (see Limits).

Accuracy

Measured on a 34-sample first-party set (8 real, 8 fabricated, 6 chimeric, 6 retracted, 6 specific-not-in-abstract) and on openly licensed third-party datasets, including the data of the July 2026 five-tool study. Per-category numbers, misses, false alarms, method and caveats: benchmark/COMPARISON.md. The samples are small — read the numbers as signals, not rates.

Limits

• The keyless claim check reads abstracts only and uses a small model (nli-deberta-v3-xsmall in the browser). Bring your own OpenRouter key for an LLM judgment instead — the more accurate path, not yet scored in our benchmark, and still abstract-only.
• "Not found" is not proof of fabrication: real papers, software, web pages and standards can be missing from the free indexes (UNVERIFIED, not FABRICATED).
• Venue, year, pages and a partly wrong author list are not checked.
• An independent July 2026 study of five hallucinated-citation checkers (Badalova & Mayr, arXiv:2607.22693) found all of them limited by extraction errors, incomplete metadata, database coverage and inconsistent results, and advised manual checks of flagged references. veripaper was not one of the tools tested, and we have no reason to think it is exempt.

Check .bib files in CI (GitHub Action / CLI)

pip install git+https://github.com/szzxli/veripaper
python -m veripaper check paper/refs.bib            # exit 1 on fabricated / chimeric / retracted
python -m veripaper check "**/*.bib" --fail-on fabricated,chimeric,retracted,unverified --json report.json

# .github/workflows/citations.yml
on: [pull_request]
jobs:
  citations:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: szzxli/veripaper@main
        with:
          files: paper/refs.bib          # space-separated paths or globs
          # fail-on: fabricated,chimeric,retracted   (default; add unverified to be strict)

Free registries, no key, no account. UNVERIFIED and RATE_LIMITED are warnings by default (a real paper can be missing from the indexes; a rate limit proves nothing); flagged entries appear as annotations and in the job summary. This uses the Python pipeline, which has fewer free fallbacks than the web app (see COMPARISON.md) — review every flag by hand.

Roadmap

Three horizons. Only Now is built; Next and Later are plans, not features. Try what is built in the live demo: https://verifycites.com/papers/?ref=gh-veripaper

Now — built

• Citation checker (web app and python -m veripaper ground): per citation, does the work exist, does its metadata match (the authors you cited vs. the record), is it retracted, and does the cited abstract support the claim (abstract-level only — see Limits).
• .bib CLI and GitHub Action: python -m veripaper check and szzxli/veripaper@main gate bibliographies in CI (see above).
• A few early, unvalidated side tools sit collapsed at the bottom of the web page; they are not the focus and are not benchmarked.

Next — planned, not built

1. Claim-level checks against open-access full text (arXiv, PubMed Central): report which parts of a claim the source supports, with quotes, never a flat yes/no (iron rule 7).
2. Close the gaps in Limits: compare year, venue and pages, and catch a partly wrong author list.
3. More free registries (e.g. PubMed, DBLP) to cut UNVERIFIED.
4. Workflow integrations: Zotero, Overleaf, and a batch mode for journal editors and writing centers.
5. A public citation-check benchmark. Today's measurement is benchmark/COMPARISON.md.

Later — vision, not built

• Idea distillation: from rough notes to a novelty and contribution statement, grounded in retrieved literature.

veripaper will not estimate acceptance chances, simulate reviewers, or ghost-write rebuttals: those are predictions or generation the tool cannot verify (iron rules 3 and 5).

Iron rules

1. Every citation must come from a real retrieval result. Fabricating citations is forbidden.
2. Honest labels: [VERIFIED] / [CITED] / [UNVERIFIED] / [STALE:YYYY] / [EXTRAPOLATED] — downgrade rather than pretend verification happened.
3. The tool assists writing; it does not replace research. Final judgment always belongs to the author.
4. The rules apply to the tool itself (dogfood): documentation, marketing claims, and tool outputs all get the same honest labels. An illustrative number is labeled illustrative — never presented as measured.
5. Lookup, not generation: the verifier generates nothing — it only queries databases (CrossRef, arXiv, OpenAlex) and reports back. Hallucinations happen when AI invents references; asking "does this DOI resolve" is a lookup, a different failure mode entirely.
6. Lightweight by design: verification is database lookups, not model training. The heaviest operation is an HTTP GET. No GPU farms required to check a citation.
7. Nuance over binary (from r/PhD feedback): a reference can be real, relevant, and still not support the specific claim. Claim-level verification must report which parts are supported — "supports the method claim, silent on the generalization, weaker sample than implied" — with quotes, never a flat yes/no. Today's [CITED] label is the honest placeholder for this until then.

Quickstart (v0.1)

pip install -e .
python -m veripaper ground draft.md --out report.md

Input: paper markdown. Output: claim-by-claim verification report.

BibTeX input. Give it a .bib instead (python -m veripaper ground refs.bib --out report.md, or paste/upload the .bib in the web app) and every entry is checked by DOI / arXiv ID / title lookup with the same labels (CITED / UNVERIFIED / FABRICATED / RATE_LIMITED + [CHIMERIC] / [RETRACTED]). Parsing happens locally — in the web app, entirely in your browser; the only network traffic is the registry lookups. A .bib carries no claims, so this mode is existence-level: no claim-support judgment and no freshness tags.

Claim support without an API key (web app)

The web app checks claims against the cited paper's abstract for free, with no key and no server:

• Deterministic checks (no model): the claim's hard specifics — numbers, percentages, quoted phrases, acronyms — and named entities (datasets, models such as CoLA, RoBERTa, ResNet-50) must appear in the abstract. Missing hard specifics give [SPECIFIC-NOT-IN-ABSTRACT] (or [POSSIBLE-MISATTRIBUTION] when the number shows up in a paper the cited work itself cites).
• A small NLI model in your browser: cross-encoder/nli-deberta-v3-xsmall (ONNX, quantized, ~87 MB, via transformers.js). It is downloaded once, on the first claim that has an abstract (not on page load), cached by the browser, and runs locally. Your draft is never sent anywhere. A keyless ABSTRACT-SUPPORTED needs high entailment, no contradictory passage and no missing specifics/entities, and is tagged [KEYLESS-NLI].
• Honest scope: abstract only, and a much smaller model than an LLM — treat it as a screen, not a verdict. Retracted or chimeric citations are never labelled supported. Untick "Keyless claim check" to skip the model.
• Optional higher-accuracy path: paste your own OpenRouter key and an LLM judges each claim against the abstract instead (bring-your-own-key; the deterministic backstop still applies).

Free sources, rate limits and caching

Every lookup goes to a free, keyless public index, in this order: Crossref (polite pool, mailto) → DataCite (also covers every arXiv paper via its 10.48550/arXiv.* DOI — arXiv's own API is not reachable from a browser) → OpenAlex → DBLP → Semantic Scholar last, only if nothing else matched. Abstracts come from the Crossref record, DataCite, OpenAlex single-record lookups and Europe PMC before Semantic Scholar.

Requests go through one queue with per-host pacing, exponential backoff with jitter on HTTP 429/503 that honours Retry-After (seconds or HTTP-date), and a per-host cooldown: when a source says "come back in 30 minutes" the app does not wait — it skips that source and moves on. Results are cached in localStorage (metadata 7 days, abstracts and judgments 30 days; rate-limited or not-checked results are never cached). A failed request — offline, CORS, timeout, HTTP 5xx, an unreadable answer — is "not checked", never "not found": only a genuine 404 or an empty search result counts as a miss, so a network failure can never make a reference FABRICATED; it shows as RATE_LIMITED (not checked) with the failing sources named. A reference entry without DOI is labelled RATE_LIMITED only when fewer than two indexes could answer; if two or more answered "no match", it is UNVERIFIED and the note names the sources that were rate-limited or unreachable. Keyless OpenAlex search shares a small daily budget per IP address (the mailto parameter does not raise it), which matters on shared networks.

Benchmark

A public citation-checking benchmark (34 samples, ground truth verified live against the registries), plus runs on openly licensed third-party datasets (benchmark/public/): benchmark/ — construction and scoring rules in benchmark/METHOD.md. Latest scored runs, with before/after numbers for the web app and a comparison with other open-source checkers (RefExists, citegate, refchecker — same samples, per-category detection, false alarms, veripaper's own misses, and the tools we could not test and why): benchmark/COMPARISON.md. (benchmark/RESULTS.md holds only a pointer; the 2026-10-20 first run is in git history.)

Version plan (not built beyond v0.1)

• v0.1: Evidence grounding (current state of this repo)
• v1.0 (Phase 1 assembly, not built): end-to-end — proposal decision → impact assessment → productization direction → evidence grounding → template generation → venue auto-optimization → reviewer simulation → venue targeting → rebuttal management → acceptance-rate review loop (learning)

Privacy

Privacy: we count anonymous page views with GoatCounter — no cookies, no personal data, no consent banner needed. Your content never goes to the counter: anything you type, paste or upload stays in your browser (except requests you start yourself, such as an AI-grading or lookup call to a provider you choose). Exactly what is sent is documented at the top of the site's analytics.js.