🎯 Good Jobs

One search box → every job board. Good Jobs is a self-hosted job search aggregator that scrapes recently posted jobs from 21 sources and streams results to your browser in real time over SSE — with AI-powered intent detection, skill extraction, vector-based related-job supplements, and a permanent Redis cache warmed every 2 hours.

🇻🇳 Vietnam is the core market — LinkedIn, ITViec, TopCV, VietnamWorks, CareerViet, TopDev, JobsGo, CareerLink, Glints, ViecOi, Indeed, TechJobs, Xóm Jobs. 🌍 Opt-in global mode covers the US, UK, and Singapore — LinkedIn, Indeed, RemoteOK, We Work Remotely, Glassdoor, USAJOBS, Dice. Just type the city; the country is inferred automatically ("nyc" → New York → US sources).

🗓️ Release Timeline

Date	Feature
2026-10-03	MCP server — mcp/goodjobs_mcp.py exposes search, recent jobs, semantic search, keyword classification, city normalization, and cache status as read-only tools for MCP clients such as Claude Desktop and Claude Code
2026-10-03	career-ops submodule — career-ops (branch feat/goodjobs-provider) vendored at thirdparty/career-ops with the goodjobs provider tests
2026-08-30	career-ops integration — career-ops added Good Jobs as a providers/goodjobs.mjs source; ongoing collaboration on integration depth (/recent-jobs, /search-semantic) and a self-hosting guide in career-ops-hq/career-ops#3521
2026-08-22	Global search — opt-in coverage for the US, UK, and Singapore (LinkedIn, Indeed, RemoteOK, We Work Remotely, Glassdoor, USAJOBS, Dice) alongside the Vietnam market
2026-04-11	CV / skill matching — paste a CV or skill list and AI intent detection maps it to a canonical job keyword, with results scored against it
2026-03-25	Summary description — background LLM summarization of long job descriptions, surfaced inline in the UI

---

🔎 The Job Boards

21 sources, one query. Every board below is scraped live on each search and merged into a single deduplicated feed.

🇻🇳 Vietnam

Board	Best known for	Scraping
LinkedIn	International companies & senior roles	Headless Chromium, full description enrichment
ITViec	IT jobs, transparent salary ranges	Static HTML, full enrichment
TopCV	Largest VN general job board	Static HTML, full enrichment
VietnamWorks	Established enterprises, HR-first	Static HTML, full enrichment
TopDev	Developer-focused roles	Static HTML, full enrichment
JobsGo	Startups & SMEs	Static HTML, full enrichment
CareerLink	Japanese-affiliated companies	Static HTML, full enrichment
CareerViet	Broad coverage across industries	Static HTML, full enrichment
Glints	Southeast Asia startups	Headless Chromium, full enrichment
ViecOi	Foreign-invested firms in VN	Static HTML, full enrichment
Indeed VN	Aggregated listings at scale	Static HTML, inline enrichment
TechJobs

🌍 Global (US · UK · SG)

Type any supported city — new york, london, singapore — and the right boards are chosen automatically.

Board	Best known for	Scraping
LinkedIn	All three markets, one scraper	Headless Chromium
Indeed	Per-country domains, huge volume	Static HTML
RemoteOK	Fully remote dev jobs	Public JSON API
We Work Remotely	Remote-first companies	Public RSS feed
Glassdoor	Salary data + company reviews	Headless Chromium, list-page only
USAJOBS	US federal government jobs	Official API
Dice	US tech contracting & full-time	Static HTML

Every posting keeps its original link — Good Jobs never republishes, it points you home.

✨ Highlights

⚡ Real-time streaming	Results arrive job-by-job via SSE as each scraper finishes — no waiting for the slowest board
🤖 AI intent detection	Paste a CV, a skill list, or a vague phrase — an LLM maps it to a canonical job keyword (with Vietnamese ↔ English translation)
🧠 Vector supplement	Related jobs from other cached keywords are retrieved via embeddings and appended on cache misses
🏷️ Skill extraction	Tech skills auto-tagged per job from title + description, rendered as pills in the UI
🔁 Self-warming cache	~30 keyword×location pairs re-scraped every 2 hours; new jobs merged by link dedup; jobs pruned after 14 days
🌐 Free-text locations	No dropdowns. "hcmc" → Ho Chi Minh City, typos corrected, country inferred from the city
🚦 Fair concurrency	A semaphore caps concurrent scrapes; user requests jump ahead of background warmups

🚀 Quick Start

docker compose up

Service	URL
Frontend	http://localhost
Backend API	http://localhost:8000
API docs (Swagger)	http://localhost:8000/docs

That's it — type a job title (or paste your CV) and a city, and watch results stream in.

📡 API

Endpoint	Method	Description
/scrape-stream	POST	Main SSE endpoint — streams cached, data, vector-results, and enrichment events
/scrape	POST	Blocking scrape (returns full job list)
/classify-input	POST	LLM intent detection: free text → canonical keyword
/normalize-city	POST	Typo correction / abbreviation expansion for cities
/suggest-query	POST	Query suggestions
/search-semantic	GET	Vector search across cached jobs
/recent-jobs, /stats	GET	VN-facing views over the cache
/cache/status, /cache/overview	GET	Cache keys with fetched_ago and job_count
/cache/scrape	GET	Trigger a scrape for a specific key
/warmup/keywords	GET / POST	Inspect / extend warmup keyword pairs
/health	GET	Health probe

🏗️ Architecture

User request
     │
     ▼
 Cache hit? ──yes──▶ Stream cached jobs instantly (event: cached)
     │                       │
     no                      ▼
     │               Vector supplement from related
     ▼               cached keywords (event: vector-results)
 Semaphore
  queue
     │
     ▼
Phase 1 ── All scrapers for the resolved country run concurrently
     │      Results streamed as each finishes (event: data)
     │
     ▼
Phase 2 ── Per-site description enrichment — VN sources (LinkedIn, TopCV,
           ITViec, TopDev, JobsGo, CareerLink, Glints, ViecOi,
           VietnamWorks, CareerViet) or global sources (LinkedIn only;
           RemoteOK/We Work Remotely return full descriptions in Phase 1,
           Glassdoor is list-page-only, Indeed enriches inline in Phase 1)
           — streamed job-by-job in background

Caching & warmup

• Results are cached permanently in Redis by (keyword, location); individual jobs are pruned after 14 days (RECENT_DAYS). Global entries are namespaced (jobs:{country}:…) and excluded from VN-facing views and warmup.
• 📝 Direct submissions: Headhunters can submit a job via POST /submissions; after admin review (/admin/submissions/*, gated by ADMIN_SECRET) it's merged into search results as source "Direct". Stored in Redis under submissions:{pending,approved,rejected}, never written into the keyword/location cache.
• Background warmup: ~30 VN keyword×location pairs every 2 hours, checked every 10 minutes. LinkedIn uses f_TPR=r7200 for incremental 2-hour windows; other scrapers refetch the full window. New jobs merge into existing caches, deduplicated by link.

Project structure

goodjobs/
├── backend/
│   ├── main.py                    # FastAPI app, routes, semaphore queue, warmup scheduler
│   └── src/
│       ├── constants.py           # HEADERS, CHROMIUM_ARGS, RECENT_DAYS, skill patterns
│       ├── models.py              # Pydantic models (Job, ScrapeRequest)
│       ├── matching.py            # Title matching, skill extraction, level tagging
│       ├── cache.py               # Redis helpers (VN/global namespacing, merge, prune)
│       ├── vector.py              # Embeddings, vector index, supplement search
│       ├── intent.py              # LLM intent detection, city normalization
│       ├── warmup.py              # Warmup scheduler, per-site enrich pipeline
│       ├── summarizer.py          # On-demand description summariser
│       ├── ratelimit.py           # Per-IP rate limiting middleware
│       ├── submissions.py         # Headhunter job-submission store (pending/approved/rejected)
│       ├── graphql_schema.py      # GraphQL schema (jobs query)
│       └── scrapers/              # One module per job board (see tables above)
├── frontend/
│   ├── index.html                 # UI layout and styles
│   ├── privacy/, terms/, contact/, admin/, post-job/
│   └── src/
│       ├── main.ts                # SSE client, fetch orchestration
│       ├── api.ts                 # Stream parser, AI classify/normalize calls
│       ├── ui.ts                  # Table rendering, modal, skill pills, badges
│       └── types.ts               # Shared TypeScript interfaces
├── thirdparty/
│   └── career-ops/                # Git submodule: career-ops (consumes this API via providers/goodjobs.mjs)
├── docker-compose.yml             # Local development
├── docker-compose.server.yml      # Production (+ cloudflared tunnel)
└── .claude/skills/                # Agent skills for common repo tasks

🛠️ Extending

Common tasks (also available to coding agents as skills):

• Add a scraper: create backend/src/scrapers/<site>.py, export from backend/src/scrapers/__init__.py, register in backend/main.py → see add-scraper
• Add a skill keyword: update _SKILL_PATTERNS in backend/src/constants.py
• Add a warmup keyword: POST /warmup/keywords or edit _WARMUP_KEYWORDS in backend/src/warmup.py
• Check cache state: GET /cache/status

☁️ Deployment

Pushing to main triggers GitHub Actions:

1. Backend — Docker image built → pushed to Docker Hub → deployed over SSH (backend.yml)
2. Frontend — Built and published to GitHub Pages (frontend.yml)

Required repository secrets:

Secret	Purpose
DOCKERHUB_USERNAME / DOCKERHUB_TOKEN	Docker Hub login
SERVER_HOST / SERVER_USER / SERVER_SSH_KEY / SERVER_PASSWORD / SERVER_PORT	SSH deploy target
VITE_API_URL	Backend API URL baked into the frontend build
CLOUDFLARE_TUNNEL_TOKEN	Cloudflare tunnel auth token

A separately-operated instance runs outside this CI/CD flow, with its own cron watchdog (restarts on DNS breakage, a crashed container, a wedged event loop, or a stalled scrape scheduler via GET /warmup/heartbeat). That script is versioned at ops/watchdog.sh for review; deploying a change to it is a manual copy to that host, not part of this repo's automation.

🤖 Agent Skills

This repo ships agent skills so coding agents (Claude Code, opencode, …) can work on it effectively:

• add-scraper — end-to-end recipe for adding a new job board
• debug-scrapers — diagnosing broken selectors, anti-bot blocks, and empty results
• cache-ops — inspecting, flushing, and reasoning about the Redis cache
• run-and-verify — start the stack and verify changes end-to-end

🧩 Third-party: career-ops

career-ops is vendored as a git submodule at thirdparty/career-ops (branch feat/goodjobs-provider). It consumes Good Jobs as a source via providers/goodjobs.mjs, a thin client that calls the public POST /scrape endpoint and normalizes the results.

# Clone with the submodule
git clone --recurse-submodules https://github.com/vnk8071/goodjobs.git

# Or, in an existing clone
git submodule update --init --recursive

Test the provider:

cd thirdparty/career-ops
node --test tests/providers/goodjobs.test.mjs

The unit tests cover URL resolution, request building, normalization, and retry behavior. They do not hit the network. The provider's default backend is https://api.goodjobs.io.vn. Because career-ops' SSRF guard blocks loopback addresses, a local docker compose up instance cannot be used as a source; point api: at a publicly reachable HTTPS deployment instead.

To update the submodule to the latest branch commit:

git submodule update --remote thirdparty/career-ops

🔌 MCP Server

mcp/goodjobs_mcp.py exposes Good Jobs to MCP clients (Claude Code, Claude Desktop, …) as read-only tools over stdio. It calls the public backend by default, so no local stack is needed.

Tool	Wraps	Purpose
search_jobs	POST /scrape	Live search by keyword and location; newest first
recent_jobs	GET /recent-jobs	Latest postings across cached searches, optionally one board
semantic_search	GET /search-semantic	Match by meaning, e.g. a pasted CV summary
classify_keyword	POST /classify-input	Turn free text or a CV into a canonical job title
normalize_city	POST /normalize-city	Fix city typos and abbreviations ("hcmc" → Ho Chi Minh City)
cache_status	GET /cache/status	Cache freshness and job counts per keyword and location

Admin and submission routes are deliberately not exposed.

cd mcp
python3 -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt
python -m pytest -q test_goodjobs_mcp.py   # unit tests, no network

Register it with Claude Code:

claude mcp add goodjobs -- /absolute/path/to/goodjobs/mcp/.venv/bin/python /absolute/path/to/goodjobs/mcp/goodjobs_mcp.py

Set GOODJOBS_API_URL to point at another deployment. Use http://localhost:8000 for a local docker compose up. The first search_jobs call on an uncached keyword can take a minute, because it runs a full multi-board scrape.

🤝 Contributing

PRs are welcome! The most valuable contribution is a new job board for your country — see CONTRIBUTING.md and the add-scraper guide.

📄 License

MIT — each job posting links back to its original source. See also our privacy and terms pages.