posturi.gov.ro scraper

→ posturi.gov2.ro [WIP]

Alternative browser / explorer for posturi.gov.ro. Scrapes the Romanian government job listings portal — and tracks changes over time. Pipeline: index → cache announcement pages → extract structured data → LLM-extracted structured display sections stored in Postgres.

Derivative work: mariuscomper.uk/posturi-publice

Project review and next work

The October 3 project audit records the code, documentation and live-site findings. Use the consolidated backlog for current priorities and the remediation specifications for bounded coding-agent assignments. Historical tracking is preserved in the activity log and the pre-consolidation backlog.

The agreed product direction captures the user-facing feature scope and recommended sequence toward an application companion.

Pipeline

flowchart LR
    web(["posturi.gov.ro"])

    subgraph scrape["① Scrape"]
        direction TB
        fetchIdx["fetch-index.py"]
        indexCSV[/"posturi_gov_ro.csv"/]
        fetchDetail["fetch-anunturi.py"]
        htmlCache[/"anunturi/\n**∕*.html"/]
        download["download-attachments.py"]
        dlFiles[/"downloads/\n*.docx *.pdf"/]

        fetchIdx --> indexCSV --> fetchDetail --> htmlCache
        htmlCache --> download --> dlFiles
    end

    subgraph parse["② Parse"]
        direction TB
        parseScript["parse-anunturi.py"]
        anunturiCSV[/"anunturi.csv"/]
        calendarCSV[/"calendar.csv"/]
        parseScript --> anunturiCSV & calendarCSV
    end

    subgraph db_layer["③ Import → Postgres"]
        direction TB
        importCSVs["import_csvs"]
        extractCmd["extract_attachments"]
        inferCmd["infer_postings"]
        pg[("jobs_jobposting\nbody_markdown\nattachment_text\ninferred JSONB")]

        importCSVs --> pg
        extractCmd -->|"attachment_text"| pg
        pg --> inferCmd -->|"inferred"| pg
    end

    subgraph enrich["④ Enrich"]
        llmSchema["llm-schema.py"]
    end

    subgraph serve["⑤ Serve"]
        direction TB
        pg[/"PostgreSQL"/]
        sqliteExport["export-to-sqlite.py\n--active-only"]
        sqliteDB[/"posturi.sqlite\n(active only)"/]
        webapp["Django admin +\nLLM variant tools\n(local, internal)"]
        phpApp["PHP webapp\n(shared hosting)"]
        browser(["browser"])
        
        pg --> sqliteExport --> sqliteDB --> phpApp
        pg --> webapp
        phpApp --> browser
        webapp --> browser
    end

    web -->|"/toate-posturile/?pg_page=N"| fetchIdx
    web -->|"/joburi/{slug}/"| fetchDetail
    web -->|"wp-content/uploads/"| download

    htmlCache --> parseScript
    anunturiCSV & calendarCSV --> importCSVs
    dlFiles --> extractCmd

    pg -->|"body_markdown\n+ attachment_text"| llmSchema
    llmSchema -->|"schema_json"| pg
    pg --> webapp

Quick start — run everything

python pipeline.py

Full update → deploy

# Scrape, parse, import, LLM enrich, export SQLite — then push to shared host
python pipeline.py && ./deploy-php.sh user@host ~/posturi.gov2.ro

# Quick refresh: skip slow steps that haven't changed
python pipeline.py --skip download,extract,infer,schema && ./deploy-php.sh user@host ~/posturi.gov2.ro

Selective runs

python pipeline.py --steps fetch-index,parse,import   # specific steps
python pipeline.py --skip download,infer               # skip slow steps
python pipeline.py --steps infer --provider gemini     # LLM inference only
python pipeline.py --no-llm --force                    # re-run dict-only inference
python pipeline.py --continue-on-error                 # log failures, keep going

Steps

Step	Script / command	Output
fetch-index	fetch-index.py	data/posturi_gov_ro.csv
fetch-detail	fetch-anunturi.py	data/anunturi/**/*.html
parse	parse-anunturi.py	data/anunturi/anunturi.csv + data/calendar.csv
download	download-attachments.py	data/downloads/
import	manage.py import_csvs	Postgres jobs_jobposting table (normalises județe — see below)
extract	manage.py extract_attachments	JobPosting.attachment_text
schema	llm-schema.py	JobPosting.schema_json JSONB
infer	manage.py infer_postings	JobPosting.inferred JSONB
occupations	normalize-titles.py	jobs_occupation + JobPosting.occupation
salary	estimate-salaries.py	JobPosting.salary_estimate JSONB
observe	manage.py observe_postings	jobs_postingobservation — immutable history of deadline, cancellation and role facts (UX-12). Optional: a failure warns and the run goes on to export; POSTURI_SKIP_OBSERVE=1 skips it

fetch-detail is a bounded refresh, not a one-shot cache: a cached page whose last successful retrieval is older than --refresh-hours (default 24) is re-fetched, at most --max-refresh (default 200) per run. Each page gets a <slug>.meta.json sidecar beside its .html (url, fetched_at, content_hash, bytes, status), written atomically. A cached page with no sidecar (pre-FIX-03) counts as due — unknown age is not fresh. Index-cancelled postings are skipped. Due pages are ranked before the cap applies — open competitions first, newest first — and a competition whose index expiry is more than --refresh-expired-days (default 30) past is not refreshed at all. The step exits 1 when more than --max-failure-share (default 0.5) of at least five attempted fetches failed, which blocks the deploy like any failed step.

When a refreshed page's Detail Content Hash differs from the stored one, import requeues what was derived from it: attachment_text is cleared (the extract step refills it from the cache) and inferred gets a stale_revision marker that infer selects and replaces — the previous values stay visible until then. Schema extraction needs no marker (--resume compares revisions itself); occupation and salary recompute every row each run. A legacy row's first hash is "unknown", not a change. Calendar events of a posting whose re-parsed page has no schedule are removed.

schema runs before infer: infer_postings reads schema_json for the announced salary, so the other order leaves inferred.salary_min one run stale.

--force re-processes already-done rows for import, extract, infer, schema and occupations, and passes --clear to salary (use it after rebuilding the grid). --limit N restricts infer, schema and occupations to N items (useful for testing). --provider gemini|openai|anthropic|deepseek sets the LLM used by the infer, schema and occupations steps (default: gemini). --no-llm skips the LLM portion of infer and drops occupations to --link-only (postings are still linked to titles already in the dictionary, which needs no calls). The schema step is always LLM-driven; use --skip schema to omit it.

Sampling a prompt before a corpus-wide run

A full run is thousands of LLM calls. --compare writes only to jobs_jobpostingschemavariant and never touches jobs_jobposting.schema_json, so a sample changes nothing the site serves and the nightly keeps using whatever LLM_PROMPT_VERSION says.

python llm-schema.py --prompt-version v4 --compare --model-filter deepseek \
    --active-only --limit 200 --workers 4
python ops/check-v4-sample.py

--limit takes the newest postings first, so a sample is reproducible and covers what the site is actually serving. --model-filter matters: without it --compare fans out to every model marked enabled in models_config.json.

check-v4-sample.py reports the deadline-extraction rate, how far expires_at overstates it, how often v4 recovers a deadline the scraper missed and whether the two ever disagree, then exits non-zero if the sample is too small, the deadline rate too low, or the output hit the token budget — truncation is silent, so it has to be checked rather than noticed. It also flags calendar rows whose label says "rezultate" but whose stage does not, which is what an enum gap looks like from the outside.

Estimated pay (salarii/, ocupatii/)

Romanian public-sector postings essentially never state a salary — 44 of 9,757 do in the body, 5 of 917 attachments — so pay is derived from the 2026 draft salary law rather than scraped.

docs/salarii/…xlsx  --build-salary-grid.py-->  data/salarii/grila-<versiune>.csv
job title           --normalize-titles.py--->  jobs_occupation (COR + grid selector)
employer            --uat.py---------------->  population band
                                    |
                                    v
                   salary_grid.estimate()  ->  JobPosting.salary_estimate

The LLM emits a grid selector, never a number. The law is an unadopted draft with more than one public variant, so re-costing under a new one must be a script run, not a re-extraction of 9,600 postings:

python build-salary-grid.py --version-id 2026-08-20   # rebuild the registry
python pipeline.py --steps occupations,salary,export-sqlite --force

data/salarii/ and data/ocupatii/ are git-tracked on purpose: every figure the site shows must stay traceable to a sheet and row, including under an older variant. python build-salary-grid.py --report prints coverage and the rows flagged for manual review.

Every estimate is gross, at gradation 0, and carries its legal status. Net pay, sporuri and the 20% cap are deliberately out of scope — see the activity log.

Județe (counties)

The source badge has two shapes — Timiş before the 2026-07 redesign, TIMIŞOARA, Timiș after — and storing both verbatim once gave the database 261 Judet rows for a country with 42 counties, which quietly broke the județ filter. webapp/apps/jobs/judete.py is now the single place that interprets a raw county string:

• normalize_judet(raw) → (county, locality), folding the Turkish cedilla ş/ţ onto Romanian ș/ț and splitting the city out into JobPosting.locality.
• import_csvs applies it on the way in, so new data cannot fragment.
• manage.py normalize_judete [--dry-run] repairs data imported before the fix. Idempotent.

When a county cannot be matched, the raw value is kept in JobPosting.judet_raw and the posting gets no county — so it answers to no county filter. Three things surface that:

1. import_csvs prints a warning listing the unmatched values.
2. judet_sanity_warnings() runs on every import (honours --strict) and fires if the Judet table exceeds 42 rows or any posting is unresolved.
3. Django admin → Anunțuri → filter Județ — rezolvare → Nerecunoscut.

The fix is normally one line in judete.ALIASES.

LLM Provider Comparison

Compare multiple LLM providers and prompt versions on the same postings without overwriting production results:

# Run all enabled models (respects enable/disable flags in models_config.json)
python llm-schema.py --compare --limit 10

# Test specific models by regex
python llm-schema.py --compare --model-filter "gemini-.*" --limit 5
python llm-schema.py --compare --model-filter "gpt-.*" --limit 5

# Test different prompt versions (when multiple versions exist in config)
python llm-schema.py --compare --prompt-version v2 --limit 10

# Combine: test GPT models with prompt v2
python llm-schema.py --model-filter "gpt-.*" --prompt-version v2 --limit 3

Each variant is stored in JobPostingSchemaVariant with:

• Provider, model, prompt version
• Token counts (input/output)
• Cost (USD) calculated from config pricing
• Latency (ms)

View results:

• Django admin: Admin → Schema LLM variants (filter by provider/model/date)
• Variant tools (Django, local only): /llm-variante/ is the per-model/prompt cost, latency and quality leaderboard; /job/<pk>/variants/ is the field-by-field matrix for one posting. Run webapp/manage.py runserver and open them directly (the public PHP site has no link to them)

Configuration (models_config.json):

• Models: enable/disable flag per model, pricing (including cache_input_cost_per_million for cache-hit billing)
• Prompts: versioned prompts (v1, v2, etc.) centralized in config
• get_enabled_models() respects "enabled": true/false flags

Prompt v2 (Schema.org-aligned)

The v2 prompt extracts a flat superset of Schema.org JobPosting properties — keys named after JobPosting properties where they exist (responsibilities, educationRequirements, experienceRequirements, qualifications, skills, baseSalary, jobBenefits, workHours, jobLocation), plus three RO-government-specific custom keys (application_docs, application_fee, application_contact). baseSalary, application_fee, and application_contact are structured dicts; the rest are markdown strings or null.

Pydantic models in schema_models.py are the single source of truth and feed each provider's native structured-output API:

• OpenAI: response_format={"type":"json_schema","strict":True,...}
• Gemini: response_schema=JobPostingExtraction
• Anthropic: tool-use with input_schema
• DeepSeek: response_format={"type":"json_object"} (loose) + Pydantic post-validation

A cacheable system prefix (instructions + 2 few-shot examples) is sent on every call so providers can hit their prompt cache — measured ~94–99% input-cache hit rate by the 2nd call on OpenAI/DeepSeek. Gemini is the exception: its implicit cache is best-effort and hit only 1 of 6 sequential v3 calls (2026-09-08), so do not budget for it there. See docs/llm-extraction-round.md.

Choosing the model, and long runs

Provider, model and prompt version resolve the same way in llm-schema.py, pipeline.py and quality_check.py — CLI flag > environment variable > models_config.json "defaults", implemented once in llm_config.py:

# .env
LLM_PROVIDER=gemini            # gemini | openai | anthropic | deepseek
LLM_MODEL=gemini-2.5-flash     # ignored if it is not a model of the selected provider
LLM_PROMPT_VERSION=v4          # v1 | v2 | v3 | v4 (occupation_v1 is the occupation step's prompt)

A backfill is ~9,600 calls per model, so the runner is built for that:

# concurrent, restartable, retrying
python llm-schema.py --prompt-version v3 --workers 8 --resume

• --workers N (default 4) — concurrent LLM calls; database writes stay single-threaded. Measured 5.13 s/post at 4 workers against 15–20 s sequential.
• --resume — re-pays only for provably stale extractions: no schema yet; a recorded source revision that differs from the posting's current content hash (both known); or recorded provenance with a different provider/model/prompt-version. Unknown is not stale — rows extracted before migration 0014 (no provenance) and rows whose page has not been hashed yet are skipped, so the unattended run never turns into a backfill. A confirmed source change still re-extracts a legacy row: import stamps its pre-change hash.
• --upgrade-legacy — with --resume, also re-extracts production rows that have no provenance. This is the reviewed, paid FIX-05-RUN backfill (python llm-schema.py --active-only --resume --upgrade-legacy --prompt-version v4 --workers 8); never put it in the cron.
• --max-attempts N (default 4) — transient errors (429/5xx/timeout) back off exponentially; invalid output gets one repair attempt that shows the model its own validation error.
• --urls-file PATH — retry exactly the affected postings: one full URL or slug per line (blank and # lines ignored). Matching is exact — a URL is compared whole, a slug against the URL's last path segment — so …-10 never selects …-100 the way the substring --slug does. It is ANDed with --force, --resume, --compare, --active-only and --prompt-version; entries that match no posting, or that another selector leaves out (already extracted without --force, expired under --active-only), are reported rather than dropped. A file with no entries is refused, never widened to the whole corpus.
• --dry-run — compute the selection exactly as the real run would (same WHERE, same order), print the selected count, the active-corpus size (expires_at >= today) and the first 20 slugs — and say so when the selection covers the entire active corpus — then exit 0. No API call, no database write, no run-log or failure-log record. Use it before any --resume/--urls-file run after a prompt or policy revision: python llm-schema.py --urls-file ids.txt --force --prompt-version v4 --dry-run.
• --failure-log PATH (default data/schema-failures.jsonl) / --no-failure-log — one JSON line per posting that fails for good, kind: "llm-schema-failure": (the run summary's), , , , , , , , , , , and — one entry per failed attempt , each with and, for a validation error, (the rejected raw value, e.g. an unlisted ). A failed posting keeps its previous and provenance. A posting that was rejected with a validation error, repaired and then gets the same record with ( = the failed attempts only; its database writes are unchanged), so the evidence shows how often the model emits a bad value overall, not only when it cost a posting. Like the path is relative to the working directory, and a write error never fails a run; dry runs never write it.

Exit status: 0 healthy (an empty selection is a healthy no-op); 1 systemic failures (HTTP errors, timeouts, anything unclassified) exceed --max-failure-share (default 0.5) of the attempted postings — or, once a run attempted at least 20 postings, all failures together do, or at least 20 selected postings had no usable content. Per-posting content failures (OutputTruncated, validation/parse errors) are reselected every run, so a small residue of them on a quiet run (a weekend has no new postings) never blocks the deploy, while a broken prompt or model change on a normal run still does; 2 a provider-wide fatal error (HTTP 401/402/403), after which no new calls are scheduled. Each model's run summary is appended to data/pipeline-runs.jsonl as kind: "llm-schema".

Every run also checks that quoted evidence/verbatim values actually appear in the posting (grounding.py) and reports unsupported quotes — a hallucination check with no extra LLM call.

Boilerplate stripping

Before sending to the LLM, boilerplate.py::strip_hg_1336() removes generic eligibility lines from HG 1.336/2022 / Codul muncii / OUG 57/2019 art. 542 (cetățenia română, capacitate de muncă, condamnări, pedepse complementare, condițiile generice de studii/vechime etc.). These appear nearly verbatim on every posting and otherwise drown the qualifications field with legal citation. The art. 35 dosar list survives intact (it's the application_docs content). Measured 13–25% input-length reduction on sampled postings; the bigger win is qualifications becoming role-specific signal (76–295 chars) instead of ~2300 chars of legal boilerplate. Toggle off with python llm-schema.py --no-strip.

Data quality

quality_check.py samples 5–10 diverse postings and runs all four pipeline layers through automated checks, then writes data/quality_report.json and a console summary table.

# Fast pass — no API calls (CSV fields, attachment extraction, dict-only inference)
webapp/.venv/bin/python3 quality_check.py --no-llm

# Full pass — includes LLM infer fallback + schema.org generation
webapp/.venv/bin/python3 quality_check.py --provider anthropic

# Check specific postings by slug
webapp/.venv/bin/python3 quality_check.py --slugs 2a66f376.doc,67438cc9.docx

Use the webapp venv because it has docx2txt, python-docx, and pypdf. PDF OCR fallback requires system tools: brew install poppler tesseract tesseract-lang (poppler provides pdftoppm; tesseract-lang installs ron for Romanian). After running, invoke /quality-review in Claude Code for a narrative assessment with root-cause analysis and recommended fixes.

Data

data/posturi_gov_ro.csv — job index, one row per listing, keyed by URL:

Field	Description
pozitie	Job title
url	Listing URL (primary key)
angajator	Employer
detalii	Details (comma-separated tags)
publicat_in	Publication date
expira_in	Expiry date
judet	County
url_judet	County filter URL
tip	Listing type
updates	Semicolon-separated log of field changes with dates

data/anunturi/anunturi.csv — one row per cached announcement:

Field	Description
Job Title	Position title
Employer	Hiring organisation
Location	County / locality
Job Level	Funcții de execuție / Funcții de conducere
Job Type	Permanent / Temporar
Employer Category	Angajator type (Primării, Instituții locale, Guvern și ministere, etc.)
Categorie	Funcție contractuală / Funcție publică
Announcement URL	Link to attached document or posting page
Main Body Markdown	Full announcement text converted to markdown
Other Links	Comma-separated attachment URLs
Nr Posturi	Number of vacancies
Contact Telefon	Phone number extracted from body text
Contact Email	Email address extracted from body text
Contact Persoana	Contact person name
Data Limita Depunere	Application deadline (DD.MM.YYYY, ora HH:MM)
Data Proba Scrisa	Written test date
Data Interviu	Interview date
Data Rezultate Finale	Final results date
Status	The detail page's own marker: Live / Anulat; blank = unknown
Detail Fetched At	When the detail page was last successfully fetched (from the sidecar); blank on legacy rows = unknown
Detail Content Hash	Hash of the fetched page, the revision llm-schema.py --resume compares against; blank on legacy rows = unknown

data/calendar.csv — flat competition timeline table, one row per event:

Field	Description
url	Announcement URL
eveniment	Event description (e.g. "Depunerea dosarelor", "Proba scrisă")
data	Date in DD.MM.YYYY format
ora	Time in HH:MM format (empty if not specified)

data/ is gitignored.

Setup

pip install -r requirements.txt

For LLM scripts and the webapp, copy .env.example to .env and fill in your API keys.

dox2md.py also requires system packages:

brew install libreoffice pandoc tesseract

Django setup (processing, admin and variant tools)

The public site is the PHP app below (PHP + SQLite). Django + PostgreSQL is the processing side: models and migrations, the importers and manage.py commands, the admin (/admin/), and two developer routes for comparing LLM extractions (/llm-variante/, /job/<pk>/variants/). It has no public browse, detail, statistics or feed pages — the duplicate Django UI was retired (FIX-12, 2026-10-16; PostgreSQL table names are unchanged because the export scripts read them).

python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
docker compose up -d db    # local PostgreSQL (see docker-compose.yml)
.venv/bin/python webapp/manage.py migrate
.venv/bin/python webapp/manage.py runserver   # admin + variant tools only
cd webapp && ../.venv/bin/python -m pytest -q  # Python suite (test database)

To preview the public site without Postgres or any API call, build the PHP test fixture and serve it (see also Local preview below):

php -r 'require "webapp-php/tests/fixtures/build_db.php"; build_fixture_db("/tmp/fixture.sqlite");'
POSTURI_DB=/tmp/fixture.sqlite POSTURI_TODAY=2026-10-03 php -S localhost:8000 -t webapp-php webapp-php/router.php

Hosting: the public site runs on shared hosting and the pipeline on a VPS (docs/deploy-vps.md). The former Fly.io files (webapp/fly.toml, webapp/Dockerfile) were removed; Fly.io is not used for production.

PHP webapp (shared hosting)

A lightweight PHP frontend that runs on commodity shared hosting (cPanel). Reads from a read-only SQLite database — no Python, no web server config, just PHP + SQLite.

Tests

The PHP app has its own fixture-based suites under webapp-php/tests/ — no network, no production data, no paid APIs. Each suite builds a deterministic SQLite fixture (tests/fixtures/build_db.php) and, where it needs rendered pages, serves the real app on the built-in PHP router with a fixed clock.

php tests/sanitize_test.php     # FIX-01: Markdown sanitizer payload battery
php tests/deadline_test.php     # FIX-02: one deadline across list/detail/feeds
php tests/request_test.php      # FIX-07: request validation, routes, feeds, filters
php tests/compat_test.php       # REV-01: new code against an older export's schema
php tests/roles_test.php        # DATA-04: role-scoped requirement search, facets and detail

npx playwright test --config webapp-php/tests/browser/playwright.config.js
                                # FIX-07: browser checks at 320/375/1280

These run locally. .github/workflows/ci.yml exists (lint + PHP suites on PHP 8.2, Playwright with pinned Chromium, alongside the Django/PostgreSQL suite) but GitHub Actions is not activated — CI was postponed by decision. The fixture query decoder is webapp-php/query.php — every request passes through validated_query() at the front controller, and ?q[]=medic (a 500 before FIX-07) is a controlled 400.

Export & deploy

# Everything: rebuild the SQLite from Postgres, push code and data
./deploy-php.sh user@host '~/posturi.gov2.ro'

# Just the PHP tree — after a template or stylesheet change
./deploy-php.sh --code-only

# Just the database, already built by pipeline.py — what the cron runs
./deploy-php.sh --data-only --no-export

# See what would move, change nothing
./deploy-php.sh --dry-run

# Push nothing; compare local vs served code/data markers (needs SITE_URL)
./deploy-php.sh --verify        # exit 1 when a code or data release is pending

Set DEPLOY_HOST, DEPLOY_PATH and SITE_URL in .env and the arguments become optional. Quote a leading ~: unquoted it expands against the local home, and the script refuses the result rather than rsyncing to a path the remote host has never heard of.

The deploy script:

1. Aborts if webapp-php/static/app.css is missing (see Stylesheet below), or if DEPLOY_PATH looks like a home directory — it runs rsync --delete
2. Runs export-to-sqlite.py --active-only — pulls active postings (expires_at >= today) from PostgreSQL into webapp-php/posturi.sqlite, unless --no-export
3. Refuses to ship a database that fails integrity_check or has no postings
4. Code: rsyncs webapp-php/ with --delete, minus assets/ (Tailwind source), router.php and tests/ (dev only) and *.sqlite* — excluding the database also protects it from the deletion pass
5. Data: rsyncs posturi.sqlite alone, no --delete and no --inplace, so rsync's write-temp-then-rename swaps it atomically and a request mid-transfer still sees the whole previous database
6. Code deploy only: stamps the current git rev-parse --short HEAD into webapp-php/static/code-version.txt (gitignored) before the rsync, so the host names the checkout it runs
7. Checks SITE_URL returns HTTP 200, then verifies the served markers on /versiuni.json — code after a code push, data.built_at after a data push — with up to 5 attempts 5 s apart, since the shared host may serve the old file for a moment. A mismatch fails the deploy. All of step 7 is skipped when SITE_URL is unset

/versiuni.json (webapp-php/feeds/versiuni.json.php, Cache-Control: no-store) reports the deployed code marker and the data provenance (built_at, run_id, git_sha, postings, active_only, detail_fetched_at_max, detail_fetched_rows, index_checked_at) as two separate facts; source_host is not exposed. --verify compares the same two markers against this checkout and the local posturi.sqlite.

Order matters across a schema change. Code and data deploy separately, so new PHP code can land on an export written by older pipeline code. db() covers the known gap: when job_postings lacks application_status (exports before FIX-05), it shadows the table with a TEMP view that computes the status exactly as the export does (shim_legacy_export() in webapp-php/db.php, tested by tests/compat_test.php). Any new column the templates start reading needs the same treatment or a VPS-first rollout: upgrade the VPS checkout, migrate, export and --data-only first, then --code-only.

The full archive stays in PostgreSQL; the deployed SQLite only contains currently active postings.

The export builds beside its target and only replaces it once it opens, passes integrity_check, clears --min-rows (default 100) and has not lost half its rows against the file it would replace. --force overrides the row floors. This is the guard that was missing when the live site served three active postings for five weeks.

Each export carries a build_meta row — when it was built, from which commit and host, and what it holds. FIX-06 added the provenance fields: run_id (the pipeline run, from POSTURI_RUN_ID), index_checked_at / index_scan_pages / index_scan_complete (from data/index-scan.json), and detail_fetched_at_max / detail_fetched_rows (newest Detail Fetched At and how many rows have one); all are empty/null on legacy builds = unknown, never synthesized from build time. The site reads it for the provenance block on /despre/ and for the tooltip behind the "actualizat" stamps; build_meta() in helpers.php returns null on an older export, and every consumer renders without it. Note the two timestamps mean different things: the visible stamp is MAX(last_seen_at), when the source was last scraped, while built_at is when the file was generated. source_host is recorded for the deploy to verify against, never rendered.

job_postings.application_status is the one vocabulary every surface filters on: confirmed_open (deadline from the concurs or the announcement, still ahead), unconfirmed (only the listing's expiry date, still ahead), closed (any date in the past) and unknown (no date). Cancelled postings are excluded from the export rather than given a status.

The export also carries llm_costs — one row per Bucharest day, provider, model, prompt version, pipeline step and source — which feeds the "Costuri inferență" section of /statistici. It is not cut down by --active-only. From the first ledgered call on (FIX-10) the rows are source = 'ledger': measured, summed from jobs_llmcall (every provider call of the schema, infer and occupations steps, retries and repairs included; unpriced_calls counts the calls with no known cost, which are not in cost_usd). Before that they are source = 'legacy': an estimate from jobs_jobpostingschemavariant, which a re-extraction overwrites and which never saw infer or occupations. Variants created at or after the first ledgered call are not exported, since the ledger already holds that spend. An export that predates the table makes the section read "Date indisponibile" instead of failing; one that predates the new columns renders with its old, "estimare incompletă" copy. See docs/deploy-vps.md § LLM call ledger.

/statistici keeps two scopes apart (FIX-09). "Export curent" counts the rows of the served file (on the host: only announcements still open at build time), split into exported / confirmed open / uncertain (unconfirmed + unknown) / closed, with the build date. "Istoric" reads publication_monthly(agg_version, month, judet_id, postings, cancelled), which export_publication_monthly() computes from every jobs_jobposting row, so --active-only and deleting expired rows never change it. month is the YYYY-MM of published_at (a source calendar date; no time-zone conversion) or unknown when it is NULL; judet_id is jobs_judet.id (0 = no county); postings includes cancelled rows (cancelled is the subset) and counts announcements, not distinct jobs. It is intake per month, not a curve of how many jobs were open. An export without the table (or with another agg_version) shows "Istoric indisponibil" and a monthly chart labelled as current-export only; db.php stands in an empty view. Reconciliation: SELECT COALESCE(to_char(published_at, 'YYYY-MM'),'unknown'), COUNT(*) FROM jobs_jobposting GROUP BY 1 must equal SELECT month, SUM(postings) FROM publication_monthly GROUP BY 1.

posting_roles (DATA-04) is the role read model: one row per advertised role, so a requirement search can demand that one role satisfies every condition. A posting whose extraction lists positions[] gets one decomposed = 1 row per entry (values from that entry only, NULL = unknown, nothing inherited from the posting); every other posting gets one implicit decomposed = 0 row that copies its flat search columns. The rules and the column meanings are in posting_roles.py (pure, no Django/psycopg); role_key is scoped to one source revision and is not stable across revisions. EQF, study level, experience, field of study, skill and credential filters (and their facet counts) read this table; everything else still reads job_postings. The export also gained job_postings.schema_source_revision / schema_prompt_version (extraction provenance). verify() refuses a build where a posting has no role row. An export that predates the table is served through a TEMP view of implicit rows (shim_posting_roles() in webapp-php/db.php), which makes the filters behave exactly as the posting-level ones did. Design and rollback: docs/specs/2026-10-10-follow-ups/02-role-requirements.md.

requirement_evidence (UX-13) holds, per extracted education / experience / skill / credential fact of each role, the quote the extraction kept and whether it can be located in the posting text the export holds (evidence.py, pure): verified (found in the page or the attachments after folding case, diacritics, punctuation and whitespace), unverified (a quote that was not found, or is too short to mean anything) or absent (no quote; says nothing about whether the requirement exists). The page shows a native <details> beside each fact; no page numbers are ever produced. attachment_text is read to run the check and is not exported. An export that predates the table shows no disclosures (shim_requirement_evidence()). The export prints coverage with denominators. Contract, limits and rollback: docs/specs/2026-10-10-follow-ups/03-requirement-evidence.md.

observe (UX-12, capture only) records what a reader may have saved a job for, because nothing else keeps an old value. manage.py observe_postings computes each posting's tracked facts (posting_facts.py: resolved deadline and its source, cancellation, expiry, per-role requirements with their evidence sentences) and inserts an immutable PostingObservation row only when the source revision, the extraction revision or the facts differ from the URL's latest observation (the first row is a baseline, not a change). It skips postings that were never successfully retrieved, so a failed fetch records nothing. --dry-run counts by class; --retain-days N prunes all but the newest row per URL (default: keep everything). No change events are derived and nothing is exported yet. Design, volume estimate, rollout and rollback: docs/specs/2026-10-10-follow-ups/06-saved-job-changes.md.

Continuous deployment

The pipeline runs unattended on a VPS twice a day (13:15 and 18:33 Europe/Bucharest) and pushes a fresh database to the shared host. The repo ships a systemd timer pinned to Europe/Bucharest (ops/systemd/posturi-pipeline.timer, needs systemd >= 240); the live box still runs a user crontab until OPS-01 is completed. Code deploys stay manual from the development machine, which is why the two rsyncs above are separate — a cron that pushed the whole directory would revert templates from the VPS's older checkout.

Mac (dev) ──git push──> GitHub ──git pull (manual)──> VPS
  │                                                    │
  │  ./deploy-php.sh --code-only     ops/run-pipeline.sh (timer / cron)
  └──────────────> shared host (PHP + posturi.sqlite) <┘

Path	What it is
ops/run-pipeline.sh	the unattended entry point: flock, pipeline, export check, data deploy, healthcheck ping
ops/check-export.py	the deploy gate — asserts the SQLite about to ship is fit to ship
ops/systemd/posturi-pipeline.{service,timer}	the two daily slots (not what the live box runs yet; the units hard-code User=posturi and /srv/posturi)
ops/env.sh	.env reader shared by the shell scripts (it is never sourced — it holds API keys)
ops/logrotate.posturi	rotation for logs/pipeline.log, installed into /etc/logrotate.d/
docs/deploy-vps.md	provisioning runbook, operating commands, failure table

Observability

Each run leaves three things behind:

Where	What
data/pipeline-runs.jsonl	one kind: "run" record (step timings, exit codes, flags) and one kind: "export-check" record (33 data metrics, every assertion), joined by run_id; plus kind: "llm-balance" (the pre-flight balance and its outcome)
logs/pipeline.log	the steps' own output, runs delimited by === posturi pipeline run <id> … ===
HEALTHCHECK_URL	/start, /fail, success — the only signal that can report a run which never happened

By default a failed pipeline step blocks the deploy: ops/run-pipeline.sh pings /fail, exits with the pipeline's status and the host keeps serving the previous database. Set POSTURI_ALLOW_DEGRADED_DEPLOY=1 to deploy anyway when the export check passes (the run is recorded as degraded).

ops/check-llm-balance.py runs first, before anything is fetched or paid for: it reads the DeepSeek balance (GET /user/balance, free) and exits 69 below LLM_BALANCE_MIN (default 1.00 USD, or when the account reports is_available: false), which ops/run-pipeline.sh turns into a /fail ping and an immediate stop. Below LLM_BALANCE_WARN (default 5.00) it only warns. A check that cannot reach the API, or a provider other than DeepSeek, prints a line and exits 0 — it never blocks a run. POSTURI_SKIP_BALANCE_CHECK=1 bypasses it; --need USD is for a manual backfill of known cost. Exit codes of ops/run-pipeline.sh: 0 fine, 65 export failed its hard checks, 69 balance too low, 75 another run holds the lock, 78 unapplied migrations.

ops/check-export.py runs between the export and the deploy. Hard checks abort before the rsync, so a corrupt export cannot reach the live site: integrity_check, broken foreign keys, duplicate URLs, a short FTS index, anything other than exactly 42 județ rows, a collapse below half the previous run, or a build_meta.built_at older than six hours (which means the export step failed and the deploy is about to ship an old file). Soft checks — coverage regressions, an altele spike, one profession family or skill tag taking an implausible share, anomaly-flag step changes, mojibake — are recorded and printed but do not block; --strict makes them fatal.

Every check is tied to a regression that actually happened here, and the two *_sanity_warnings() functions from import_csvs.py are called rather than restated, so "too many counties" has one definition in the tree.

.venv/bin/python ops/check-export.py --no-log              # read the live numbers
.venv/bin/python ops/check-export.py --db /tmp/copy.sqlite --no-log

--no-log keeps an interactive read from becoming the baseline the next real run compares against. For the narrative pass over the run log — missed slots, recurring step failures, metrics sliding while every individual check still passes — run /pipeline-check in Claude Code. See docs/pipeline-quality-checks.md for the full watch-list and what is still unbuilt.

Full setup — packages, seeding Postgres and the scrape cache from the Mac, SSH keys, installing the timer — is in docs/deploy-vps.md.

Stylesheet

The app has no build step on the host — the compiled CSS is committed. Rebuild it whenever you touch a template's classes:

npm install          # once
npm run css          # webapp-php/assets/app.css -> webapp-php/static/app.css (minified)
npm run css:watch    # while editing templates

Fonts (static/fonts/*.woff2) and htmx (static/htmx.min.js) are self-hosted; nothing is fetched from a CDN at runtime.

Skins

The whole palette is one :root block of CSS custom properties in webapp-php/assets/app.css. Tailwind's theme resolves every colour, radius and font utility to one of those variables, so re-declaring them under [data-skin="<id>"] restyles the entire site without touching a single utility class. That is all a skin is.

Three ship today, chosen with the picker in the footer (stored in localStorage, applied before first paint so there is no flash):

id	what it is
hartie	the built-in default — parchment, Fraunces, DM Sans. No file; it is :root.
govuk	GOV.UK Design System — white, square, Arial, yellow focus, black masthead.
posturi	the official posturi.gov.ro — navy + gold, Manrope, rounded, lifted cards.

Adding one: copy webapp-php/static/skins/_template.css to <name>.css and it appears in the picker on the next request — no registry, no build step. The id is the filename; inc/skins.php discovers it and reads the display name from the @skin comment. Files starting with _ are skipped.

Two rules the template explains at length:

1. Scope every rule under [data-skin="<id>"]. All skin files load on every page, so an unscoped rule leaks into the others. @font-face is the exception — it declares a family rather than applying one, and is how a skin ships its own typeface.
2. Colours are space-separated RGB channels (245 240 232), not hex. That is the only form Tailwind's alpha modifiers compose with; a hex value silently breaks every bg-surface/70 on the page.

Skins live in static/ rather than assets/ because they need no build — Tailwind would strip them, since nothing in the PHP references their selectors — and because deploy-php.sh excludes assets/.

Validate before committing:

php webapp-php/assets/check-skins.php

It reads the token contract out of app.css and flags the four failures that are silent in a browser: an unscoped rule, a token name that does not exist, a colour written as hex, and any text/background pair under WCAG AA.

Previewing another export

db.php honours a POSTURI_DB environment variable, so a test or a preview can point at a different SQLite file without touching the deployed one:

POSTURI_DB=/tmp/other.sqlite php -S localhost:8000 -t webapp-php webapp-php/router.php

Unset in production.

Local preview

php -S localhost:8000 -t webapp-php webapp-php/router.php

router.php exists only for PHP's built-in server — it serves static files and mirrors the .htaccess deny rules. Apache never loads it.

Structure

Path	Purpose
index.php	Front controller — routes /, /job/123-slug/, /angajatori/, /statistici/, /despre/, /robots.txt, /sitemap.xml
db.php	PDO singleton for posturi.sqlite
helpers.php	Markdown rendering, date formatting, filter builder, facet queries, FTS query builder, active-filter chips, URL helpers
pages/	List, detail, employer list, employer detail, stats, about
feeds/	Atom, JSON API, iCal endpoints — follow the active filters; ?employer=<slug> scopes a feed to one angajator (linked from each employer profile)
partials/	Result list and facet sidebar (both HTMX out-of-band swappable)
inc/	Header/footer HTML, <head> metadata (canonical, OpenGraph, JSON-LD hook)
assets/	Tailwind source — not deployed
static/	Compiled app.css, self-hosted fonts, htmx
router.php	Dev-server router — not deployed
.htaccess	URL rewriting, asset caching, blocks direct access to *.sqlite*