open-jobs

Open Jobs is no longer running. The crawler, the nightly build and the search page were shut down on 10 October 2026. Running them cost more than a free project with no business model could carry. The last snapshot, built 5 October 2026, stays downloadable under Take the data, and the code is here for anyone who wants to run their own. Everything below describes the project as it ran.

Every listing wears its verdict

Job boards know which listings are dead and will not tell you, because their revenue depends on a shelf that looks full. Open Jobs crawls the career sites themselves and records the day it first sees every posting. That clock cannot be re-stamped. A model learns how long each kind of posting typically lives, and every job is graded against par:

	verdict	what it means
🌱	Fresh	younger than postings like it. The market moves fast here. Apply now.
🥀	Stale	an outlier survivor. Hard to fill, which can mean thin competition.
🔁	Re-stamped	claims to be newer than when the crawler first saw it. Caught.
👻	Ghost	open for over a year. The dataset holds 40,000 postings over five years old.

Fresh is what you see by default. The rest is one click away, labelled.

Try it in your browser

The search page at backend.dehnbostele.workers.dev is offline since 10 October 2026; the address now serves a notice and the data files. This is how it worked. Type a job title, where you want to work, and a few sentences about the job you want. The page writes the posting, finds its neighborhood among 3.1 million openings, and shows you the fresh ones first. Say yes or no; a yes pulls in more jobs like it, and Rerank re-sorts the list when you ask, never underneath you. No account. Nothing except what you type is sent to the server.

Use it with your agent

The agent workflow needed the live index (/embed and the nightly groups) and stopped working with the shutdown on 10 October 2026. The description below is kept for the record. Paste this into Claude Code, Codex, Cursor, or whatever you use:

Clone https://github.com/elliottdehn/open-jobs to help me find a job.

Or install it as a Claude Code plugin:

/plugin marketplace add elliottdehn/open-jobs
/plugin install job-search@open-jobs

Your agent interviews you, writes the description of the job you want, pulls the nearest openings onto your disk, and hands you a shortlist of fresh, eligible matches ranked by how well they fit. On a laptop it also builds a local search page that learns from your yes and no. Five minutes, start to shortlist. Your labels, notes, and resume never leave your machine. You need uv; everything is uv run tools/jobs.py <cmd>.

Agents: you MUST read AGENTS.md in full before doing anything else in this repo.

Cloud sessions run behind a network allowlist that, by default, reaches package registries and GitHub and nothing else. The tools here make one kind of outside call, to the public index at backend.dehnbostele.workers.dev, and it will be blocked until you allow it. Once:

1. Open claude.ai/new#settings/capabilities.
2. Scroll to the bottom. Under Additional allowed domains, add exactly backend.dehnbostele.workers.dev (no https://, no slash).
3. Start a fresh session and paste the line again.

That is the only domain this repo needs. If your session runs in a custom cloud environment with its own Allowed domains list, add the same domain there instead.

In ChatGPT, a plain chat can't clone a repository or run its tools. If it says it can't clone open-jobs, switch the mode selector from Chat to Work and paste the same line again.

How it works

1. Crawl. One Cloudflare Durable Object per career site, 65,000 of them, each wakes at its own time of day, diffs today's listings against yesterday's, and pulls the full description once for every new job. First seen is written once and never changed.
2. Grade. A nightly batch folds the fleet into one dataset, trains the estimators (posting age, salary, seniority, work arrangement), and publishes the search index: a few thousand groups of similar jobs, each with full text and an embedding of every posting.
3. Search, locally. Your description of the job you want is embedded once, its nearest groups come down to your machine, and everything after that, ranking, labels, learning, happens there.

Around 11,000 lines of code. Design and the daily workflow: backend/DOCS.md. Enrichment fields: backend/FIELDS.md.

Take the data

The snapshot is frozen at the last nightly build, 5 October 2026, and stays downloadable while the bucket is paid for.

One link: open-jobs-latest.tar, about 31 GB, every posting open on 5 October 2026 with its full description, a 1536-dim embedding, and company fields, as one parquet file per applicant tracking system. Resumes if interrupted.

Or three commands, which also give you a query shell over it:

git clone https://github.com/elliottdehn/open-jobs
cd open-jobs
uv run tools/jobs.py export

Needs uv, one line to install: curl -LsSf https://astral.sh/uv/install.sh | sh (or brew install uv; on Windows winget install astral-sh.uv).

Three commands and the whole dataset is on your disk: every open posting with its full description, a 1536-dim embedding, and company fields, as one parquet file per applicant tracking system (~13 GB, resumable). Then uv run tools/jobs.py sql "SELECT title, company, location FROM jobs LIMIT 20" queries it with DuckDB. /data/ lists every published file with sizes, build dates, and how to read each in place.

The corpus is published as static files: a manifest (a tree of a few thousand groups of similar jobs, with labels and exemplars), centroids, and one JSON per group with titles, companies, locations, URLs, full description text, and float32 vectors. tools/jobs.py fetch pulls any set of groups into a local parquet you can query with DuckDB. The API also answers per-posting questions: POST /status with a list of ats/slug#id keys returns open or removed with timestamps, straight from the crawler's records.

Mirror the search index

Several people copied the whole search index every night while it ran. This was the contract; the files of the last build are still there, and nothing under /data/ will change again.

What it is. Three kinds of static file under /data/:

• manifest.json: the tree. Every node has an id and a list of children; the nodes with no children are the groups. The manifest also carries groups (this build's prefix, for example groups/2026-09-09/), built_at (milliseconds since the epoch), and jobs_total.
• centroids.bin: one float16 unit vector per node, 1536 dimensions, in the same order as the tree. A query is routed to groups by cosine against these.
• <groups prefix><id>.json: one file per group, named by the node id. Each holds the postings in that group: title, company, location, URL, first seen, the full description, and a base64 float32 vector per posting.

How to copy it.

1. Fetch manifest.json.
2. Fetch centroids.bin.
3. For every node with no children, fetch <groups prefix><id>.json.
4. Once a day, fetch the manifest again. If built_at changed, repeat from step 2.

Why a copy is always one build. Each build is published under its own dated prefix, and the manifest switches to it once, after every file is in place. Files under a prefix are never modified, and the previous prefix stays for a day. So a copy that starts from one manifest is one consistent tree, even if a publish happens while you are copying. Use the prefix from the manifest; the bare groups/ path exists only for readers written before the dated layout.

Size and pace. 11,572 files, 52 GB, median 3 MB (as of 2026-09-11). No group file holds more than about 2,500 postings, roughly 30 MB: an oversized neighbourhood is published as numbered chunks that share one centroid. There is no rate limit on /data/ and egress is free, so copy at whatever pace suits you.

Shortcut. uv run tools/jobs.py fetch --groups 0 does the same walk into a local parquet with the group id on every row. Node 0 is the root, and --groups fetches every group under the nodes you name.

Two asks. Put a contact in your User-Agent; two mirrors already do, and it turns a bad publish into an email instead of a mystery. And know that you may not need a mirror at all: every read on /data/ allows any origin, so a browser can route a query through the centroids and fetch the few groups it needs straight from here.

History is published too. Every day's diff against the day before, one row per event with the full job record (added, removed, changed, and the previous version of changed), sits under /data/diffs/, each with a lite/ twin that drops the vectors and keeps the text only where a mirror needs it, and a daily ledger of every job the crawler has ever recorded, open or removed, with its first-seen and removed dates, under /data/ledger/. /data/diffs/index.json and /data/ledger/index.json list what is there. Layout, schema, and endpoints: backend/DOCS.md.

The ideas channel

⏰ The repo ships a permission (.claude/settings.json) that lets an agent post short improvement notes (file:line, idea) to a shared Slack channel. That permission is a technical default, not consent: the agent gets to work without asking, and the first time it has an idea worth sharing it asks whether it may post it. On a yes it posts; on a no it immediately opts the person out with uv run tools/optin-ideas.py --out. Nothing about the person, their JD, labels, or data is ever posted. The wording and rules are in AGENTS.md under "#multipenny-ideas"; an agent that posts without asking is misbehaving.

Agent permission systems (correctly) refuse to let an agent widen its own permissions, but do let it narrow them. If the rule weren't there, saying yes would mean the person editing a settings file by hand; with it there, saying no is one command the agent can run itself, and saying yes costs nothing. The default is on so that the only action ever left to the agent is the safe direction.

License

See LICENSE.