Safety-Aware Route Planner (prototype)

An in-car GPS route-planning prototype over real OpenStreetMap data (Berkeley / North Oakland) that minimizes travel time while penalizing two unsafe maneuvers:

1. Unprotected left turns onto busy streets (permissive signal or no control — not a protected arrow)
2. Uncontrolled crossings of busy streets (straight-through where the crossing has no signal and is not an all-way stop)

For an origin, destination, and departure time it returns up to three curated alternatives — fast / balanced / safe — each with distance, ETA, and unsafe-action counts split by type, plus per-segment safety-tier coloring.

Architecture

ingestion/   OSMnx v2 download -> clean -> tag extraction -> compact binary pack
core/        C++17 routing engine (pybind11 module `sr_core`)
pyref/       pure-Python reference engine — the parity twin of core/
sim/         deterministic synthetic time-of-day traffic profiles
api/         FastAPI service (POST /route, GET /geocode) over the engine
commute/     commute planner service; so far the beta trip-trace ingest (ADR-0018)
web/         Next.js + React + MapLibre GL front-end (OpenFreeMap tiles)
config/      config.toml — every tunable number lives here
tests/       unit, safety-rule, scenario, contract and parity suites

Key design points:

• Edge-based turn graph: search state = directed edge; turn costs enter g at transitions. Dijkstra + A* (admissible time-only heuristic).
• Shared numpy precompute: all floating-point math (penalties, haversine, λ-scaling, congestion) happens once per query in pyref/costs.py; both engines only ADD prebuilt float64 arrays → Python↔C++ parity is bitwise (asserted by tests/test_parity_cpp.py).
• Cost model (all weights in config/config.toml): raw = w_man·sev + w_speed·norm(speed) + w_lanes·norm(lanes) + w_vol·norm(vol) − w_med·median, control override (protected/roundabout → 0, all-way stop → ×0.25, give-way straight/right → ×0.35 and give-way left → ×0.6, right-of-way straights/rights → ×0.2, signalized left → ×0.35, inferred-unprotected → ×0.5), penalty_s = k·max(raw,0), g = time + λ·penalty. The same post-override raw drives the unsafe-action counters (raw ≥ τ + per-type predicate + OBSERVED control) and the safe/caution/unsafe tiers.
• Alternatives: λ sweep (0 / 0.5 / 1.5) → Jaccard dedup (0.8) → penalty-method rerun for genuine diversity, with a guard that drops a diversified route if it is safety-worse than the un-inflated optimum. Labels are assigned after dedup: lowest-λ survivor = fast, highest = safe.
• Detour budget (detour_budget_pct, default 0.25): how far out of the way the safe route may go for a safer crossing, as a fraction of the fastest route's time. λ alone cannot express this — it trades penalty against time at a fixed exchange rate with no notion of how far the user will actually go. The budget works both ways: when λ will not buy a detour the budget allows, a hard-avoid run forces it; when λ produces a route longer than the budget allows, that alternative is dropped. Each route reports detour_pct.
• safety_enabled=false: λ forced to 0, single fast route; unsafe counters are still computed and reported.
• Traffic: deterministic synthetic hourly profiles by road-class group; the departure-time snapshot is frozen per query and feeds both congestion (travel time) and the volume term of the safety score.

Build & run

Prereqs: Python 3.12+, Node 20.9+, MSVC (for the optional C++ core on Windows). The 3.12 floor comes from the pinned numpy and scipy, which both require it.

Run every command below from the repository root (the directory containing this README). npm --prefix web install resolves web/ relative to your current directory, so running it from anywhere else — including from inside web/ itself — fails with ENOENT: … web/package.json.

The C++ core and a locally-built graph pack are not required for everyday front-end/API work: sr_core falls back to the pure-Python engine with a warning if it isn't built, and the API downloads the prebuilt packs it serves automatically at startup (see Pack distribution below). Use the quick start below unless you're changing core/ or ingestion/.

Quick start (macOS / Linux)

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Then, in two terminals, from the repository root:

# terminal 1 — API on :8000 (fetches its packs automatically at startup)
.venv/bin/python -m uvicorn api.main:app --port 8000

# terminal 2 — web on :3000
npm --prefix web install
npm --prefix web run dev                  # http://localhost:3000

Quick start (Windows)

python -m venv .venv
.venv\Scripts\pip.exe install -r requirements.txt

Then, in two terminals, from the repository root:

.venv\Scripts\python.exe -m uvicorn api.main:app --port 8000

npm --prefix web install
npm --prefix web run dev                  # http://localhost:3000

Full dev setup (C++ core, local packs, tests)

Windows has a one-shot script; there is no macOS/Linux equivalent yet, so mirror its steps manually with requirements-dev.txt, pip install ./core, and python -m ingestion.build_pack --region <name>.

powershell -File scripts\build_all.ps1   # venv, deps, packs, sr_core, tests

The engine implementation is chosen by [engine] impl in config/config.toml ("cpp" falls back to pyref with a warning if sr_core isn't built). scripts/bench.py compares the two (~20× speedup, <1 ms per query on the Berkeley/Oakland pack).

Regions are presets in [region.presets]; build a pack with python -m ingestion.build_pack --region <name>. Raw OSM downloads are cached in data/cache/ so Overpass is hit once per region. Cache filenames carry INGEST_SCHEMA_VERSION (ingestion/download.py) — bump it whenever tag retention or simplification changes, or a stale cache will keep feeding the old data into new code.

Pack distribution

Packs are build outputs, not source: data/ is gitignored, and the API cannot start without one. packs.lock pins which prebuilt artifacts a checkout, a CI run, and a deployed container should all use.

At startup the API downloads any missing pack (api/packs_fetch.py), verifies it against the digest in packs.lock, and unpacks it atomically. This is a no-op when the pack is already on disk, so local development never touches the network — and neither does the test suite, which pins SR_PACK_DIR at a generated toy pack.

While base_url in packs.lock is empty, fetching is disabled entirely and the API reads local disk only. To enable it:

1. Create a public-read R2 (or S3) bucket. Packs derive from public OpenStreetMap data, so there is nothing secret in them: the runtime fetches over plain HTTPS with no credentials, and only CI — which uploads — needs keys. Set the repo variable SR_PACKS_BASE_URL and the secrets R2_ACCOUNT_ID, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY plus the variable SR_PACKS_BUCKET.
2. Run the build-packs workflow (manual dispatch — it hits Overpass, so it never runs per-push). It builds each region, publishes reproducible <region>.tar.gz artifacts under an immutable tag, and prints the packs.lock stanza.
3. Paste each stanza line under [regions] in packs.lock and open a PR. Rolling forward is merging it; rolling back is reverting it.

Each stanza line carries its own tag, and the lock's top-level tag is only the default for entries without one. So one region can be published, or republished, under a new tag with a one-line paste that leaves every other region's entry and URL alone. That is how san_francisco was added next to the older Berkeley packs:

gh workflow run build-packs.yml -f regions=san_francisco -f tag=packs-v2-20260925 -f publish=true

Archiving is deterministic — sorted entries, normalized mtime/uid/mode, fixed gzip header — so packaging the same pack directory twice gives byte-identical output. The pipeline is not reproducible end to end, though: ingestion/pack.py stamps created_utc into every manifest, so two builds of identical OSM data yield different pack bytes and different digests. The digests in packs.lock therefore have to come from the run that published the artifacts; they cannot be re-derived by rebuilding. scripts/package_packs.py does the packaging and can be run locally.

To pull published packs into a fresh checkout without building from Overpass:

python -m api.packs_fetch --regions berkeley_small,berkeley_oakland,san_francisco

The API serves the packs in [api] regions in config/config.toml: berkeley_oakland and san_francisco, with the first as the default (ADR-0014). SR_REGIONS=berkeley_oakland narrows a deployment to one.

Running it (including on a phone)

The container

docker build -t sr-api .
docker run -p 8000:8080 sr-api

Dockerfile is multi-stage: the builder installs g++ and produces an sr_core wheel, and the runtime stage carries no compiler, no ingestion/, and no osmnx — that package and its geo stack are build-time only, and excluding them keeps the image at ~520 MB instead of well over a gigabyte. The build fails if sr_core cannot be imported, because pyref/engine.py otherwise falls back to the pure-Python engine with only a warning, and that is a ~20× latency regression whose only symptom is slow responses.

The served packs are baked in at build time, fetched from the public bucket pinned in packs.lock (see Pack distribution) and checked against its digests, so a deployed machine boots without depending on the bucket. Build with --build-arg PACK_REGIONS= to skip that, and the container downloads them at boot instead.

/health reports readiness, not just liveness: 503 until every served pack is loaded, and it includes engine so a silent downgrade to the pure-Python path is visible from outside without reading logs.

{"status":"ok","packs_loaded":2,"regions":["berkeley_oakland","san_francisco"],"num_edges":48841,"engine":"cpp"}

Serving both packs takes about 130 MiB of container memory at idle, measured as cgroup memory.current. berkeley_oakland alone takes about 91 MiB (ADR-0014's 2026-09-24 amendment).

On a phone, via a tunnel

Geolocation needs a secure context, so GPS origin tracking will not work over plain http from a phone — you need HTTPS even for local testing. A tunnel is the cheapest way to get it, and needs no hosting account.

The front-end proxies /api/* to the backend (rewrites in web/next.config.ts), so one tunnel to port 3000 serves the whole app. That matters more than it looks: NEXT_PUBLIC_API_URL is inlined at build time, so pointing the browser straight at the API would mean rebuilding the front-end every time a tunnel handed out a new hostname. Going through the proxy also means no CORS configuration at all.

# terminal 1 — API on :8000
docker run -p 8000:8080 sr-api

# terminal 2 — web on :3000, proxying /api to the container.
# The PRODUCTION build, not `run dev` — see below.
npm --prefix web run build && npm --prefix web run start

# terminal 3 — public HTTPS URL for :3000
cloudflared tunnel --url http://localhost:3000

Open the printed https://….trycloudflare.com on the phone. API_PROXY_TARGET overrides the proxy destination if the API is somewhere other than http://localhost:8000.

Serve the production build, not next dev. The dev server's hot-reload websocket lives at /_next/webpack-hmr, and Next blocks cross-origin access to dev resources by default — so over a tunnel the upgrade is refused and cloudflared logs

unable to reach the origin service … malformed HTTP response

on a loop. The app still works; it is hot reload failing noisily. Adding the tunnel host to allowedDevOrigins would silence it, but quick-tunnel hostnames change every session, so you would be editing config each time. A production build has no HMR client at all, starts faster on a phone, and is closer to what would actually ship.

On Windows the cloudflared MSI installs to C:\Program Files (x86)\cloudflared\ without adding itself to PATH, so either call it by full path or add that directory to your user PATH.

If you deploy it later

The MVP runs on Fly.io (route API and commute service) and Vercel (web/); the runbook is docs/deploy.md. Nothing here is host-specific, though — it is a container that reads its configuration from the environment (SR_CORS_ORIGINS, SR_CORS_ORIGIN_REGEX, SR_PACKS_URL, SR_REGIONS, SR_NOMINATIM_CONTACT, SR_REDIS_URL, SR_TRUSTED_PROXIES). Four constraints carry over to any host:

• HTTPS is mandatory, for the geolocation reason above.
• Scale out with replicas, not --workers. A single process already uses every core: the C++ search releases the GIL and api/routes.py is a sync handler, so FastAPI runs it in a threadpool. Extra uvicorn workers add no throughput and duplicate the loaded graph pack in RAM once per worker. More capacity means more containers behind a load balancer.
• Set SR_REDIS_URL before raising the replica count above one. The Nominatim budget is a token bucket (api/ratelimit.py, ADR-0013). With a Redis URL it is one bucket for the whole deployment, so the ~1 req/s ceiling holds however many replicas there are. Without one it is per process, and N replicas means N requests per second against a service whose policy allows one — so autoscaling without Redis still needs an explicit max of 1. Over budget, GET /geocode returns 429 with a Retry-After; if a configured Redis is unreachable it returns 503 rather than serving (ADR-0013). SR_RATE_LIMIT_KEY separates deployments that share one Redis instance.
• Set SR_TRUSTED_PROXIES if anything sits in front of the container. POST /route and POST /reroute carry a per-client quota (30/min, burst 15 — far above anything the front-end can emit, see config/config.toml), and with no auth yet the client is a network address. Behind a load balancer every peer address is the balancer, so the quota silently becomes one bucket for the whole deployment and any single caller can throttle everyone. The fix is X-Forwarded-For, but that header is caller-supplied: it is only readable if you know how many proxies appended to it. Set this to the true hop count (1 for a balancer, 2 for CDN-then-balancer). It defaults to 0, meaning the header is ignored entirely — blunt on purpose, because the alternative default lets anyone mint unlimited identities by varying a header. Over quota, both endpoints return 429 with a Retry-After; if a configured Redis is unreachable they serve anyway, which is deliberately the opposite of the geocode answer above (that ceiling protects a third party's policy, this one protects our own CPU — see the 2026-09-18 amendment to ADR-0013).

API contract (frozen — a future mobile client reuses it)

POST /route {origin:{lat,lon}, destination:{lat,lon}, departure_time, safety_enabled, detour_budget_pct} →

{ "routes": [ {
    "kind": "fast|balanced|safe",
    "geometry": { "type": "LineString", "coordinates": [...] },
    "distance_m": 0, "eta_s": 0, "detour_pct": 0,
    "unsafe": { "unprotected_left": 0, "uncontrolled_crossing": 0, "total": 0 },
    "segments": [ { "geometry": {...}, "tier": "safe|caution|unsafe" } ],
    "unsafe_points": [ { "lon": 0, "lat": 0, "type": "unprotected_left|uncontrolled_crossing" } ]
} ] }

detour_budget_pct (request, optional — config default when omitted) and detour_pct (response) are the additive fields; everything else is unchanged.

POST /reroute {origin:{lat,lon}, destination:{lat,lon}, preference} → { "route": { ...one route artifact... } }. Replans from the current position to the original destination carrying a prior artifact's preference, and recomputes only that one safety level (ADR-0008) — so a nav consumer stays at its chosen level instead of silently swapping onto whichever level is fastest from the new position. Additive endpoint; the /route contract is unchanged.

Both routing endpoints draw on one per-client quota (they spend the same resource — a graph search), and over it they return 429 with a Retry-After and FastAPI's standard {"detail": ...} body. No response field was added; the frozen shape above is untouched.

GET /geocode?q=... proxies Nominatim (rate-limited, identified UA, cached) bounded to the pack bbox. Set SR_NOMINATIM_CONTACT to override the User-Agent contact string per deployment. Over the shared ~1 req/s budget it returns 429 with a Retry-After rather than queueing the caller — the front-end's 600 ms debounce and the response cache absorb it.

GET /meta → {region, bbox, num_edges} — additive endpoint so the client can tell whether a GPS fix falls inside the routable region. bbox is [west, south, east, north], or null for packs with no declared coverage.

Front-end behavior

• Current location. The origin tracks live GPS (watchPosition). A deliberate origin — map click, marker drag, or geocode pick — disables follow mode so it is never overwritten; the ⌖ button re-enables it. Re-routing is throttled to movements over 25 m and debounced, so GPS jitter doesn't hammer the router. A fix outside the pack bbox falls back to the default view with an explanation rather than a 422.
• Units. Miles by default, toggleable to km, persisted in localStorage. The API always speaks SI (distance_m, eta_s); conversion is presentation-only so the response contract stays framework-agnostic for a future mobile client.
• Map robustness. MapLibre only auto-resizes on window resize, so a map created before layout (or while the tab/pane is hidden) would strand its canvas at the 400×300 fallback inside a full-size container — a blank map. A ResizeObserver plus a timer-based reconciliation (ResizeObserver and requestAnimationFrame callbacks are both tied to the page's rendering steps, which a non-compositing page never runs) keep the canvas matched to its container, and map errors surface in a banner instead of failing silently.
• Live navigation (experimental, off by default). With NEXT_PUBLIC_ENABLE_LIVE_NAV=1, a Start navigating button appears once a route is selected against a live GPS origin. It swaps the planner for a turn-by-turn HUD (next maneuver, distance/ETA remaining, optional spoken instructions, and upcoming-hazard banners). Leaving the route triggers a reroute at the same safety level (POST /reroute carries the route's preference, so a reroute can never silently drop you from fast onto safe), the new route replaces the old one in place, and reaching the destination ends the trip with a You have arrived panel. Without the flag the whole mode is unreachable — the planner behaves exactly as before.

See docs/map-provider-tradeoffs.md for the MapLibre vs Google Maps Platform evaluation.

Trying live navigation on a desktop

Real turn-by-turn can't be exercised by walking around at a desk, so a dev-only mock GPS driver feeds a scripted track. Start the web server with the flag on (from the repository root):

NEXT_PUBLIC_ENABLE_LIVE_NAV=1 npm --prefix web run dev

Then, in the browser console at http://localhost:3000:

// 1. Drop a live fix inside the Berkeley/North Oakland pack so the origin
//    goes "live" (⌖ button lights up). Click the map to set a destination;
//    routes appear and the Start navigating button shows.
__srMockGeo.start([{ lat: 37.8715, lon: -122.2680 }]);

// 2. Click Start navigating, then play a track that follows the route and
//    then veers off it to see the reroute. One fix per second:
__srMockGeo.start(
  [
    { lat: 37.8715, lon: -122.268 },
    { lat: 37.8700, lon: -122.266 },
    { lat: 37.8690, lon: -122.262 }, // <- off-route: HUD shows "Recalculating…"
  ],
  { intervalMs: 1000 },
);

__srMockGeo.stop(); // halt the track

window.__srDebug exposes the live routes/selection/progress for inspection. Both handles are stripped from production builds. For a real GPS test on a phone you need HTTPS — see On a phone, via a tunnel above.

Intersection control: observed vs inferred

Whether a maneuver is unsafe depends almost entirely on what holds back the traffic you cross, so control classification is the accuracy-critical step.

OSM maps stop signs and traffic signals on the approach arm at the stop line, not on the junction node — and OSMnx simplification deletes those interstitial nodes along with their tags. On berkeley_oakland that discarded 87% of highway=stop nodes, 25% of highway=traffic_signals, and 100% of highway=give_way, so most intersections fell through to a road-class guess and fully signalized crossings were reported as uncontrolled.

ingestion/approach_controls.py therefore harvests control from the unsimplified graph, walking outward from each junction along every arm (up to [ingest] max_control_offset_m) and resolving per approach:

• a signal on any arm signalizes every leg (pedestrian crossings excluded — see _is_pedestrian_signal);
• a stop node on every arm is an all-way stop, even without stop=all, which is nearly never tagged;
• otherwise must_stop is read off which arm the stop node sits on, rather than guessed from road class;
• a give_way is ignored at a junction that has stop signs — a junction is stop-controlled or give-way-controlled, not both, and the stray tag is usually a yield-to-pedestrians marking (198 of 308 give-way junctions in berkeley_oakland are this mixed case);
• evidence is harmonized across the two arms of one named street, because a device that governs a road governs both directions of it. Tagging one arm only is a mapping gap; left alone it makes one direction of a through street "must stop" while the opposite direction keeps priority, which sends routes off that street mid-block;
• direction / traffic_signals:direction / stop:direction are honored.

Anything the tags do not settle still falls back to the road-class heuristic in ingestion/controls.py. The two cases are distinguished by edge_control_confidence (ControlConfidence.OBSERVED / INFERRED), and only OBSERVED approaches can be counted as unsafe maneuvers or painted red — a guess is not evidence of a hazard. Inferred unprotected approaches still carry a damped penalty (inferred_confidence_factor), so routes keep preferring known-controlled intersections without the UI claiming the unknown one is dangerous. approaches with OBSERVED control in the sanity report is the headline number: ~56% on berkeley_oakland.

Use scripts/inspect_junction.py --lat --lon to see how any single intersection was classified and why.

Modeling notes / heuristics (documented approximations)

• Permissive-vs-protected signals are not tagged in OSM, so every signal is SIGNAL_PERMISSIVE; SIGNAL_PROTECTED arises only via the control_override hook (tests/future data). A left at a signal is therefore discounted (signal_left_factor) into the caution band rather than counted: a signal is a form of traffic control, and routing toward signalized intersections is the point.
• "Busy" is a tunable weighted score of speed/lanes/volume ([busy]), so busyness is time-of-day dependent. "Major" additionally requires the road to be physically big (major_lanes_min / major_class_max) and gates the stop-controlled predicates. In practice busy almost implies major — only 14 of 3,698 busy edges in berkeley_oakland are busy-but-not-major — so the gate is a safety net rather than a load-bearing filter.
• A 2-way stop offers the approach that must stop no protection at all: both the left onto and the straight across a major road are counted. The approaches with priority through the same node are not.
• Median presence is inferred from one-way major-class edges (dual carriageways map as one-way pairs).
• The crossing-street busyness of a STRAIGHT maneuver is approximated by "any other incoming approach at the node is busy".
• U-turns are disallowed except at dead ends (fixed penalty, config).

Non-goals (scope guard)

No live traffic, no accounts, no ML scoring; OSM turn-restriction relations are an optional future hard-constraint module. Live turn-by-turn navigation (with spoken instructions) exists but is experimental and off by default behind NEXT_PUBLIC_ENABLE_LIVE_NAV — see Live navigation above. Mobile is future scope — the API contract is framework-agnostic for a React Native/Expo or PWA client.