Spideryarn

AI-assisted reading that augments rather than replaces reading. Live at www.spideryarn.com; this is the code behind it.

it augments human cognition, but it doesn't replace it … help the user get what they need from it, help them read efficiently, but deeply, help them internalize and interrogate.

— Greg, 2026-08-24

Everything in this repo is downstream of that sentence.

What it is

A companion, not a replacement: it highlights, annotates, orients and explains, but keeps you in the text itself. Or, as the front page puts it, it's like reading a dog-eared copy of a book where a clever friend has highlighted the best bits and scribbled in the margins to help with the difficult bits — rather than reading the Reader's Digest version.

It is for people who read difficult material and think professionally: scientists, researchers, academics, editors, anyone who opens a long, deep, important article — a paper, a philosophy essay, a policy report — that they want to understand, digest, internalise, critique and remember.

The problem, and the bet

Nearly every AI reading tool makes the same move: compression. Paste an article, get bullet points, done. That is genuinely useful for triage and genuinely corrosive for understanding: a fluent impression of the piece and none of its texture, no argument you could reconstruct, no sentence you could quote. The summary replaced the reading instead of supporting it.

you can go to the gym or you can buy a forklift truck to lift the weights. But if you buy the forklift truck that lifts the weights, then you atrophy. … our goal is to make things easier where we can, but not too easy.

— Greg, dictated notes, 2026-09

So the bet is to make deep reading cheaper, not optional. Scan the landscape quickly. Descend on demand into the actual prose, at the point you care about. Stay oriented at whatever altitude you are flying. Ask questions at the moment of confusion, in place. Come away with something retained.

Three commitments separate this from the summariser it superficially resembles (vision.md § Principles):

• The text is the destination, not the source material. Every generated line is a door into the prose, not a wall in front of it. We never silently rewrite the author's words: the full text is always there beside whatever the model produced.
• When the AI says what the article says, it cites the passage. One press and you are reading the author, not the model. Nothing it asserts is left unanchored.
• It is a reading tool, not a writing or chat tool. The article never leaves the screen. And when a design call is genuinely close, the tiebreak is: which option will best help the reader form their own rich, updated internal representation — digest, understand, learn, notice, integrate, critique?

And deliberately not, ever (vision.md § Anti-goals): "read this in 2 minutes"; streaks, nudges, anything optimising for time in the app; confident generated claims with no path back to the source.

What it does today

This started as an experiment and is a real product now, with accounts, and billing live since 2026-09-03. It is a beta with very few readers, so we still optimise for how fast we can move.

What it does is help you read deeply and efficiently. Greg, 2026-10-09:

reading deeply means like understanding it, learning, remembering, having ideas, noticing, highlighting, perhaps seeing problems, seeing connections, perhaps to your own work, being a referee for a journal. These are all examples of deeply.

And then efficiently. … it takes less time and it works better than just reading the original paper.

It gets there by several paths rather than one: the marked-up copy from a clever friend; that friend to talk to and bounce ideas off; knowing who you are and why you are reading this piece, so that what you get is relevant to your purpose; AI that helps you rather than doing it instead of you; and evidence underneath, the learning-and-memory research behind Learn above all.

The article is always on screen. Beside it is a band that the reading modes take turns in (reading-view-overview.md is the map). The ones marked experimental are behind a switch on /profile (experimental-features.md) because they are not yet good enough to show a stranger by default.

Finding your way around the piece

• Plain — the prose and nothing else, the default. Every other mode is a step away from it and a step back.
• Structure — the piece's parts and sections as a tree: two linked columns in a wide window, a nested list in a narrow one (structure.md). It is what is left of the first feature, granularity zoom, and the same tree drives the Spine down the side of the page (granularity-zoom.md).
• Summary — the piece in plain words, brief or fuller, each paragraph linked to the passages it came from, or the same as a numbered thread (summaries.md, tweets.md).
• Skim — a route through the paper's key passages, walked with a handful of stops, then a dozen, then more (skim.md).
• Marginalia — notes in a column to the right of the prose, each level with the paragraph it is about: a question for each part, where an idea first appears, and where the argument has got to (marginalia.md).
• Diagram (experimental) — the shape of the argument as a picture a model draws, and optionally paints (diagram.md, sketch.md).
• Search by meaning — a word, a phrase, or a description of what you are after; the relevant passages are marked in the prose, each hit saying how sure it is (search.md).
• A command bar — type or say what you want ("jump to the first place it mentions X") and it opens the right mode on the right passage ().

Understanding it

• Glossary — the terms this piece uses in a non-obvious way, defined from the piece itself and underlined wherever they occur (glossary.md).
• Ideas — the propositions the piece assumes you already hold, and the ones it introduces (ideas.md).
• Quotes — the most central, striking lines, in the author's own words (quotes.md).
• Cross-references and links — a phrase that sums up another passage is underlined and jumps to it (cross-references.md); hover the author's own hyperlinks and see where they go before you leave (links.md).
• FAQ (experimental) — the questions a careful reader would ask, each answered by the piece's own passages rather than by a written answer (faq.md).
• Sources — every work the piece cites, what others say about it, and checks of the claims it rests on (sources.md).
• Timeline (experimental) — when the piece says things happened, showing the uncertainty rather than hiding it (timeline.md).

Asking

• Select a sentence — that bookmarks it; optionally add a note, or ask for an explanation, which streams in while the article stays on screen (comments.md).
• Chat — ask a longer question; whenever the answer says what the article says, it links to the passage. It does not summarise the article unless you ask it to (chat-tools.md). You can also talk to it out loud and switch between speaking and typing (live-conversation.md).
• Referee (experimental) — for peer reviewers: your own criteria checked against the paper with every matching passage marked, and a mirror that rereads your draft comments. It helps you read the paper without reading it for you (referee-mode.md).

Keeping it

• Learn — say what you took from the piece and find out where it holds and where it comes apart (learn-mode.md); or the other way round, a Quiz in which the article asks and you answer, marked against the text rather than an answer key (quiz.md).
• The library — every article you have added, one click from where you left off (library.md). Make one public and strangers can read it at /read/public (public-shelf.md), or share it by a private link; your notes, chats and searches stay yours.
• It knows who is reading — say once who you are and what you know, and for any article why you are reading it, and the notes are written for you (reader-profile.md).
• It's the reader's data — one button exports everything Spideryarn holds about an article as plain files (export.md); what we do with a reader's data is in privacy.md. If you have a checkout of this repo, an AI agent such as Claude Desktop or Claude Code can also use your account through a small MCP server that runs on your machine (mcp.md).

Chat deserves a flag, because "a chatbot with the article stuffed in the context window" is a named anti-goal here. It was built anyway, at Greg's request, and the argument that what was built is not the anti-goal — along with an honest account of where that argument is weakest — is in 260826a-chat-mode.md § Say the awkward thing first.

Pricing

Reading is never charged for; only adding an article spends a slot. The free tier is three articles, for life, with no card required; Reader is $10 a month (£8 · €9) for twenty a month, and Researcher $50 a month (£40 · €45) for a hundred and fifty. An article you make public costs half a slot. The live figures are src/web/PlanCards.tsx § WEBSITE_PLANS and /pricing; what a slot is and why it is priced this way is billing.md.

Running it yourself

You need Node 26, Docker running, and an OpenRouter API key — nothing else, no accounts on anybody's dashboard. The commands, in the order that works, with a "did it work?" check after each and what the failures mean, are in setup-dev.md § Quickstart; the rest of that file has the per-stage pipeline commands, which model does which job, and what each secret is for.

How it works

docs/tutorials/architecture.html — The shape of Spideryarn — explains the whole system to somebody who knows the product and has never opened the code: how an article is fetched, cut into blocks and retold at several lengths, why every paragraph gets a permanent id so your notes survive a re-import, and why the pipeline builds a new revision beside the live one rather than editing in place. Open it in a browser. Three more go a level deeper: import-pipeline-and-database.html, revisions-and-the-schema.html, and 260906a-deployment-and-infrastructure.html — Shipping Spideryarn — which follows one commit all the way to the live site and is the place to start if you want to know what it takes to run this thing.

For engineers, architecture.md is the reference, and block-ids.md is the one contract everything else depends on — read it before touching anything that resolves an id.

Where the real documentation is

This file is a signpost, and so is AGENTS.md (CLAUDE.md is the same file). Everything real lives in docs/project/, and AGENTS.md has the full table — seven entry-point docs, one line per doc beneath each, saying what is in it. Start there. The handful worth naming here:

Doc	What's in it
vision.md	what we're trying to do, the principles, and what we're deliberately not doing
reading-view-overview.md	everything the reader sees, and every mode
setup-dev.md	install, dev, every pipeline command, and which model each job uses
original-version/	the larger app this came from — a library to consult, not a backlog to import

docs/plans/ holds the reasoning behind work being done or just done; docs/postmortems/ one file per bug worth understanding; docs/research/ the options weighed behind a decision. None of them are indexed — list the folder and read the file names.

Contributing

Contributions are welcome, and the most useful one is not code.

The easiest and best way to help is the Feedback button, top-right of the reading view once you are signed in. A really well-described bug report or feature request — what you were doing, what you expected, what happened instead, and why it mattered to your reading — feeds directly into the product-building pipeline. Reports are stored, triaged, and worked through in batches, with a note kept under docs/user-feedback/ and the reasoning written down (feedback-reports.md is the process; 260904b is what one batch looks like). Nothing here has produced more change per minute spent than a precisely-described report.

And if you want to write code, I'd love to work with you. Get in touch first at [email protected] — say what you want to build and we'll work out whether it fits, which saves you building something the vision doc rules out (vision.md § Anti-goals is worth a read either way). setup-dev.md takes an empty checkout to a running app, and the architecture tutorial explains how the whole thing works.

One unusual condition on pull requests. If you built it with an AI agent — and most of this repo was — include the prompts and the conversation alongside the diff. Greg, 2026-09-06:

If someone wants to submit a pull request, they have to also include the prompts/conversation with their agents that helped them build it, so I can understand their intent/approach.

Paste them into the PR description, or add them as a file under docs/plans/. This is not a formality: in a codebase where the code is cheap and the intent is the scarce thing, the conversation is the part that says what you were trying to do and what you decided not to do — which is exactly what a reviewer needs and a diff never shows. It is the same reason every piece of work here keeps a plan doc.

CONTRIBUTING.md is the short version of this section, plus one thing in the tests that looks like a leaked key and is not.

Working here

Read AGENTS.md — it's the working agreement, for people as much as for agents. The short version: several sessions work this tree at once, so stay inside your stage, never run a git command that throws work away, and commit your own files by name (version-control.md has the recipe and why it has the shape it does).

Licence: MIT, covering this project's own code and documentation and not the third-party articles kept as test inputs under evals/ and tests/fixtures/.