Housing & Economic Data

A multi-tab static website of Canadian housing & economic data, built for a commercial real-estate appraiser — CMHC rental market, housing starts, secondary rental, census housing & demographics, affordability, Manitoba rent control, and market indicators — with copy-to-Word, Excel (.xlsx), and PNG exports for appraisal reports. The Rental Charts tab shows interactive line charts of CMHC rental and vacancy data, filterable by geography (Province / CMA / CSD / Survey Zone / Neighbourhood), dwelling type (Apartment / Row / All), and a breakdown dimension (Bedroom Type / Year of Construction / Structure Size / Rent Ranges).

Map views. Several tabs render self-hosted choropleth maps (Observable Plot; no external tiles, so the strict CSP and PNG/Word export stay intact): municipality choropleths on Affordability, Census Profile, and Housing Stock (shaded by a chosen metric; click a municipality to select it), and a survey-zone / neighbourhood picker map on Rental Charts. Boundary GeoJSON is built by r/20_build_boundaries.R (StatsCan cartographic boundaries) and r/21_build_cmhc_zone_boundaries.R (CMHC survey geographies) and committed under web/public/data/geo/, so the running site fetches static files and a fresh clone needs no boundary rebuild.

Geographic coverage. Manitoba, Saskatchewan, Alberta and British Columbia carry full detail — every level (province, CMA/CA, survey zone, neighbourhood, and municipality/CSD). Manitoba has the deepest history (rental data back to 1990); the other three cover the most recent 15 years. Every other province/territory includes its major centres (province + CMA/CA level) for a rolling recent window. The rental/starts geography list is config-driven in r/lib/cmhc_helpers.R (the PROVINCES table, detail = full | basic); CMA/CA UIDs are derived from the cmhc package's translation table. Each geographies.json item carries a prov code, which drives the Rental Charts province-first filter (Province → Level → Area, scoped to one province at a time).

Tabs

The header carries two top-level groups, each with its own bar of category menus below it (web/src/nav-menus.js; each menu's trigger names the tab open under it, and every tab keeps its #hash link):

• Housing & Economic Data (public; the landing view): Rental (Rental Charts, Rental Tables, Compare Areas, Secondary Rental, RTB (MB)) · Housing (Housing Starts, Housing Stock, Affordability, Parade of Homes) · Economy & People (Census Profile, Current Snapshot, Market Indicators, Construction Cost, Commercial Leasing, Agriculture). Construction Cost and Commercial Leasing are catalog groups tagged tab (rendered by web/src/cost-lease-tabs.js, still tiled on the Snapshot via inSnapshot), each with an indexing tool: a cost or rent from a year/quarter brought to the latest quarter. Winnipeg BCPI starts 2017Q1, so earlier years chain onto a proxy (NHPI house-only for residential; a chosen 15-CMA / Calgary / Edmonton / Prairie BCPI for non-residential); Winnipeg CRSPI chains onto Canada CRSPI before 2019, shown beside CPI.
• Local Data (tabs whose data comes from folders on the viewer's disk): Cap Rates · Cap Rates vs Interest · Market Discount Rates · Multifamily (Yardi Rental, Rentals.ca, Rent Comparison by Source, Rental Dashboard) · Commercial (Johnson Report, Altus Costs, Brokerage Reports, Commercial Dashboard) · FCC Farmland · Zoning. Stand-alone tabs sit on the bar as plain cmhc-tab buttons. Market Discount Rates and Altus Costs are the exceptions: their figures are published, so they ship with the site.

To add a tab, put its tab-btn-* button in the right menu panel in web/index.html and list it in setupTabs (and TAB_GROUP if it is Local Data) in web/src/main.js.

The Local Data folders all live in one Dropbox folder shared with colleagues, AppMarketData (Jason's copy: D:\Dropbox\SharedInfo\AppMarketData), holding BrokerageReports, CapRates, CommercialAvailability, JohnsonReport, RentalData, RentalDashboard, Zoning and SalesData (the last read by the two parcel-search sites). Each tab accepts its own subfolder or AppMarketData itself: web/src/app-market-data.js (resolveAppFolder) steps down to the tab's subfolder before the handle is saved, and filterAppFiles does the same for the folder-<input> fallback.

One pick connects every tab: the strip under the Local Data menu bar (web/src/market-data-bar.js, logic in web/src/market-data.js) saves a handle to AppMarketData, or to any folder holding some of its subfolders, and fills each tab from it. A tab with no saved folder gets its subfolder saved, and an empty tab gets it imported. A folder picked on the tab itself wins, and a tab that already holds data is left to its own update check. Several folders can be connected, for colleagues shared only some subfolders, which Dropbox puts at the top of their Dropbox. On later visits Chrome needs one "Allow" on the connected folder, which covers everything inside it. Open tabs and the two dashboards reload through web/src/market-data-channel.js (a BroadcastChannel). SalesData is not in SOURCES and is never opened. A new folder-backed tab needs a SOURCES entry, a saveDirectory export on its store, and an onSourcesUpdated listener. TAB_GROUP in web/src/main.js says which tabs belong to Local Data; every other tab is Housing.

Tab	What it shows	Pipeline
Rental Charts	CMHC Rms line charts (5 metrics) + survey-zone picker map	r/01–r/04, r/21
Rental Tables	Appraisal-ready vacancy / median-rent pivot tables + bar charts	r/03
Compare Areas	Multi-area time series within one province	r/03
Secondary Rental	CMHC Srms (condo / secondary) for surveyed centres	r/06
Housing Starts	CMHC Scss starts / completions	r/05
Housing Stock	Census dwelling type / age / condition + choropleth map	r/07–r/10, r/12b, r/20
Census Profile	Population & dwelling trends + annual estimates + demographics + community-profile narrative + choropleth map	r/12, r/12b, r/20, r/26, r/27 (run-once); r/23 (monthly)
Affordability	Royal-LePage-style affordability factor + choropleth map	r/16, r/18, census, r/20
Current Snapshot / Market Indicators	BoC / StatsCan / OSB economic indicators, incl. CPI inflation (headline, core, shelter, annual average), (Cap vs Interest moved to its own Local Data tab)	r/10,r/11,r/13,r/14,r/17
Cap Rates vs Interest	The Cap vs Interest chart — BoC overnight target and GoC yields with Winnipeg cap-rate overlays from the Cap Rates tab's local data (Local Data group)	BoC shard + Cap-Rates project
Market Discount Rates	Canada's market risk-free rate, market risk premium and required return to equity from Pablo Fernandez's annual surveys (2010 on; RF and Km from 2017, plus 2013 and 2015): headline tiles for the last year in range, the workbook's table, line and stacked-bar charts (Local Data group, but the data ships with the site in web/public/data/economy/market_discount_rates.json). After a new survey, add its row to Market Risk Premium Survey Chart.xlsx beside the study PDFs and run the script, which cross-checks the newest PDF and records each study's title/authors/date. Below that, a "Comparison to other investments" section: growth of $100 in XIC (S&P/TSX total return), ZFL (long federal bonds) and CPI over an adjustable window (effective date, years back), a summary table, and an auto-written discount-rate narrative (Copy / Word) citing the newest survey and Steinbach / Access Credit Union 5-year GIC rates — refreshed weekly

The Tables tab generates appraisal-ready comparison tables (vacancy / median rent by bedroom type, rent range, year built) with copy-to-clipboard (rich HTML for pasting into Word), Word (.docx), and Excel (.xlsx) export — this replaces the retired CMHC-VacancyMedianRents Shiny tool. It is province-scoped (pick a province — always the first row, default Manitoba — then the second–fourth areas are centres within it; no cross-province comparison), and each table has a grouped-bar chart card beside it (same chrome as the Rental Charts cards — title, subtitle, right-side legend, Download PNG). The Compare Areas tab is the multi-area counterpart to Rental Charts: it overlays several areas within one province as time-series lines for a fixed breakdown category (e.g. pre-1960 stock across MB centres), with a matching areas × years table beside each chart — one chart+table pair per metric (median rent, average rent, vacancy, avg rent change). The pipeline also pulls the Secondary Rental Market Survey (Srms — condo rental data) into web/public/data/secondary.json, replacing the retired "CMHC Rental Data Scrape" project. Both retired projects are archived under $Projects in Progress\old projects maybe.

Cap Rates vs Interest (Local Data)

The Cap Rates vs Interest tab (formerly the first section of Market Indicators) puts the BoC overnight target and the 5- and 10-year Government of Canada yields on one axis with average commercial cap rates by property type, so the risk premium between them is visible directly. It is the web version of the IntChartALL / IntChartO / IntChartMF family from the "Interest Rate Charts" Quarto project (D:\Dropbox\Appraisal\RProjects\appraisal-templates\$ Interest Rate Charts), collapsed into one card whose overlays are chosen with checkboxes instead of being baked into fifteen PNGs — the checkboxes and the CSV picker sit in the column beside the chart, not above it. It follows the tab's existing date range, and both halves are resampled to monthly means so the daily rate series and the quarterly cap rates share x-values.

No cap-rate data ships with the site. The rate lines come from the committed BoC shard; the cap rates are the same local data the Cap Rates tab reads — cap_rates.json from the firm's SharedInfo\AppMarketData\CapRates folder, held in this browser's IndexedDB by cap-rates-store.js — so picking that folder once on the Cap Rates tab serves both tabs and there is one cap-rate source to maintain. (The earlier separate Colliers CSV upload here is gone.) The section has a firm picker — Colliers by default, or CBRE, Cushman & Wakefield, or the average of all three — and one overlay per property type (Multi-Family, Office, Industrial, Retail on by default; Hotel and Self Storage available), each the mean of that firm's published class mid-points for the quarter, with Office covering all four downtown/suburban classes. The section listens for the Cap Rates tab's hed:cap-rates-updated event so a fresh import shows without a reload. Because the section is not shard-backed it is rendered by its own module rather than the catalog loop; cap_vs_interest appears in displayGroups only so the sidebar toggle and jump link pick it up.

Rental Dashboard and Commercial Dashboard (sub-apps)

The Manitoba Rental Explorer and the Commercial Availability Explorer, formerly their own Vercel sites, live in this project as two more pages — web/rental/ and web/commercial/ (each with its own index.html, src/, tests and README, copied from their source projects' web/ folders) — built by the same Vite config (build.rollupOptions.input) and shown inside their Local Data tabs in an iframe. Same origin is the point: the folder picker and IndexedDB work inside the frame exactly as they did standalone, and the viewer connects the same SharedInfo\AppMarketData\RentalDashboard / CommercialAvailability folders. The frame's src is set on the tab's first visit so the map stack (MapLibre + Protomaps + pmtiles, ~900 kB) is not fetched until asked.

Their static assets moved under web/public/rental/data/, web/public/commercial/data/ (the two apps' boundary GeoJSONs differ, so they are kept apart) and the shared web/public/basemap-sprites/. The CSP in vercel.json gained the basemap hosts (pub-…r2.dev, demotiles.maplibre.org), worker-src blob: for MapLibre's tile worker, frame-src 'self' blob: and frame-ancestors 'self' (with X-Frame-Options: SAMEORIGIN) for the frames. npm run test:subapps runs their node --test suites; npm run lint covers their sources. Their pipelines (RentalDashboard, Commercial-Availability) are unchanged and still publish to the shared folders. When updating a sub-app, edit it here; the source projects' web/ folders are now the historical copy.

Chart captions and the Company name

Every indicator chart (Market Indicators, Agriculture, Cap vs Interest) carries its source in the subtitle and the company name in the caption under the figure, the way the reference appraisal chart does — so a chart pasted into a report is attributable on sight. The header box is the only place the name is set; it is saved per browser as it is typed (firmName in the shared prefs, web/src/firm.js) and read back on the next visit, so it only has to be entered once. Every open card and the PNG each one exports pick it up live. It starts empty — an unset field signs nothing — and clearing it goes back to unsigned charts. The CMHC-sourced cards (Rental Charts, Tables, Starts, Housing Stock, Compare Areas) keep Source: CMHC in the caption, where the licence attribution has always been.

Built as a Vite + vanilla JS static site, with an R-based data pipeline. Deployed to Vercel from this GitHub repo.

Parked features

Features that are built and kept current, but deliberately out of the navigation. Nothing here is deprecated — each is a candidate to finish and re-expose, and each keeps refreshing on the scheduled workflows so it does not rot while parked.

Feature	State	How to bring it back
MB Economic Update tab	Hidden from the nav since 2026-08 (also hidden 2026-06, revived 2026-07). The panel, web/src/economic-update.js, and the r/15 + r/16 pipeline are all intact and still refreshing; the tab is still reachable directly at #economic.	Delete the hidden attribute on #tab-btn-economic in web/index.html.

The MB Economic Update tab generates an automatic narrative report — StatsCan economy indicators, the CREA HPI benchmark, and a manual outlook section — and is a likely candidate for future build-out rather than removal.

Architecture

• r/ — R pipeline (uses the cmhc package by mountainMath) that produces long-form JSON shards under web/public/data/.
• web/ — Vite static site (vanilla JS + Tailwind v4 + Observable Plot).
• vercel.json — points Vercel at the Vite build, and rewrites /gh-data/* to the shard proxy.
• api/gh-data.js — edge proxy that fetches shards from the data repo (below) with Vercel's edge cache in front of raw.githubusercontent.
• .github/workflows/refresh-data.yml — monthly cron that re-runs the R pipeline, publishes shards, and commits refreshed JSON.

The shards live in a second repo

The per-geography CMHC shards — series/ (Rms) and starts/ (Scss), ~401 MB across 1,340 files — are not in this repo. They live in housing-economic-shards and are fetched at runtime through /gh-data/....

Why: Vercel stores the full build output of every retained deployment separately, with no de-duplication between deployments. With the shards here, Vite copied all 401 MB into dist/ on every build, so changing a single shard cost another ~425 MB of deployment storage. This project alone was ~25 GB of the team's 36 GB. Moving them took the build output to ~25 MB.

web/src/shards.js pins the exact commit the app reads. Pushing shards does nothing until that pin moves — which is the point: production is never exposed to a half-pushed tree, and a bad refresh is rolled back by restoring the previous SHA rather than by re-uploading data. refresh-data.yml publishes and bumps the pin automatically; it needs a SHARDS_TOKEN secret (a PAT with contents: write on the shards repo) and fails loudly if the shards changed and no token is set.

The rest of web/public/data/ — indicators, housing, economy, geo, the manifests, ~24 MB — deliberately stays here: several parts are read before first paint, so the extra hop would not pay for itself.

Local development

# 1. Install web dependencies
cd web
npm install

# 2. Pull CMHC data and build JSON shards (one-time / on refresh)
cd ..
npm --prefix web run data:all

# 3. Dev server
npm --prefix web run dev

Open the URL Vite prints (usually http://localhost:5173).

Refresh data

Three options:

1. Wait for the scheduled GitHub Actions runs (see schedule below).

2. gh workflow run refresh-data.yml (full CMHC refresh) or gh workflow run refresh-indicators.yml (indicators only) to trigger manually.

3. Local: npm --prefix web run data:all && git add web/public/data && git commit -m "data: refresh" && git push.

   Note that this publishes everything except the series/ and starts/ shards, which are gitignored here and live in the data repo (see Architecture above). To publish those by hand: copy web/public/data/series and web/public/data/starts into a checkout of housing-economic-shards, commit and push, then set SHARDS_REVISION in web/src/shards.js to the new commit SHA and push here. Until that pin moves, production keeps serving the previous shards.

Census Profile tab (run-once, separate from data:all)

The Census Profile tab (Population & Dwelling Trends 2006–2021 + a Demographics comparison across three user-chosen areas at a selectable census period, the web port of the MBCensusData report) is built by r/12_census_profile.R into web/public/data/housing/census_profile.json. The tab has three free area pickers (any Manitoba geography or Winnipeg Community Area / Cluster / Neighbourhood) feeding both tables, and a Census-period selector. The period drives only the Demographics table + demographic charts; the Trends table/chart always span all censuses. demo is keyed by census year (2021 plus best-effort 2016 & 2011 — period-of-construction buckets and the income reference year shift each census, and 2011 long-form is the NHS, so those earlier years are fetched leniently and some rows blank out). It uses StatCan census via CensusMapper / cancensus (a different source than the rest of the pipeline), so it is deliberately excluded from data:all and from the GitHub Actions refresh — census data is 5-yearly, and CI has no CensusMapper key. Re-run it manually only when a new census is released:

# CensusMapper key (see MBCensusData/"Cancensus API Key.R") — pass via env:
$env:CM_API_KEY="CensusMapper_xxx"; npm --prefix web run data:census

The tab also auto-writes a community-profile narrative (growth, income, dwelling values, housing stock, employment by industry — Area 1 read against Areas 2/3 as benchmarks, with a Word download; web/src/census-narrative.js, see docs/neighbourhood-narrative-plan.md). Its "Employment by industry" paragraph and table come from r/26_census_industry.R → web/public/data/housing/census_industry.json: labour force aged 15+ by the 20 NAICS sectors, 2021 + 2016, for PR/CMA/CD (MB/SK/AB/BC) and every Manitoba CSD. Same CensusMapper key and run-once rules as r/12; it is in census-refresh.bat. Set CENSUS_INDUSTRY_WPG=1 to also aggregate the Winnipeg Community Areas / Clusters / Neighbourhoods from the 1,130 City DAs (2021 only — most of a free-tier daily quota, resumable through the shared cache). The narrative simply omits the paragraph for any area/period the file lacks.

Its "Access and roadways" paragraph (municipalities only) comes from r/27_build_csd_highways.R → web/public/data/geo/mb_csd_highways.json: the Manitoba Road Network 2023 (Manitoba Transportation and Infrastructure via the Manitoba GeoPortal, Manitoba Open Data Licence; cached under r/lib/cache/geo/) filtered to PTH + PR routes, dissolved by signed number and intersected with the shipped mb_csd.geojson — routes through each CSD, routes within 15 km (distance + compass direction), and the boundary distance and direction from Winnipeg. No key; run-once (npm --prefix web run data:highways); re-run only if the road network or CSD boundaries change.

Its "Services and amenities" paragraph is the one manual input: a Services checklist under the narrative (municipalities only) where the appraiser ticks what the community has; the rubric in web/src/service-tier.js (amenity list + tier rules: full-service regional centre / full-service / limited-service / minimal-service) makes the call and writes the sentence, naming the nearest larger centre for whatever is missing. Ticks are saved per municipality in the browser (localStorage) only — nothing is published. Rural POI data is too patchy to automate this; the definitions are the asset.

Separately from the run-once census build, r/23_scrape_population_estimates.R (part of data:all and the monthly refresh) pulls StatsCan's annual July-1 population estimates (table 17-10-0155, 2001–present, 2021 boundaries) for every western municipality and province into web/public/data/housing/population_estimates.json — the Census Profile tab renders these as a "Population — annual estimates" chart, giving a between-censuses trend for any selected province or municipality.

A companion script, r/12b_wpg_city_history.R, then appends earlier-census history (2006/2011/2016) to the Winnipeg clusters + community areas from the City of Winnipeg's published census profiles (Community Social Data Strategy custom tabulation) — CensusMapper DA-aggregation only yields 2021 for those virtual geos. It needs no API key (downloads from legacy.winnipeg.ca, cached under r/lib/cache/, gitignored) and uses the crawled neighbourhood→ cluster→CA manifest r/lib/wpg_city_neighbourhoods.csv. 2016 comes from the clean per-cluster/per-CA .xlsx; 2011/2006 are summed from per-neighbourhood .xls (counts only — medians aren't aggregatable, so they're dropped for 2011, and 2006 is trends-only). It also pulls dwelling condition (% needing major repairs, incl. a City 2021 pass since CensusMapper has no condition vector) so the Housing Stock tab can offer the Winnipeg clusters/CAs alongside the standard geographies. It also grafts the City's 2006/2011/2016 per- neighbourhood profiles onto the WPG_Nbhd geos (matched by name — ~175–180 of 211; the rest are industrial/zero-population areas with no City profile, plus a few that changed names/boundaries between censuses), giving neighbourhoods a 2006→2021 trend (2006 trends-only) + 2011/2016 demographics. Run it after r/12 (census-refresh.bat does both):

Rscript r/12b_wpg_city_history.R   # no key; appends WPG cluster/CA 2006–2016

Coverage: all Manitoba PR / CMA-CA / CD / CSD geographies, plus the City of Winnipeg virtual geographies (Community Area / Cluster / Neighbourhood, dissemination-area aggregated via r/lib/wpg_geography_lookup.csv; clusters and community areas additionally carry 2006–2021 trends + 2011/2016 demographics from the City of Winnipeg, and most neighbourhoods carry 2006–2021 trends + 2011/2016 demographics from the City's per-neighbourhood profiles). Free CensusMapper keys are capped at 500 region identifiers/day (5,000/month), and Winnipeg has ~1,130 dissemination areas — more than one day's allowance — so the DA pull is chunked (18 DAs/request) and the cache is persistent, making the run resumable: re-running on later days replays cached chunks for free and pulls ~500 more, so the full Winnipeg build completes over ~3 days (or in one run with a higher quota from CensusMapper's maintainer). If the daily/monthly quota is exhausted mid-run the script still writes all standard Manitoba levels and skips the Winnipeg virtual geos with a warning; re-run later to add them. Standard Manitoba levels are cached after the first build, so those re-runs are free.

Find by address (Winnipeg)

The Census Profile sidebar has a Find by address box that resolves a Winnipeg street address to its Neighbourhood / Neighbourhood Cluster / Community Area, loads the cluster as Area 1, and offers the other two levels as one-click alternates. It is entirely offline — no geocoder, no API key, no external request — so the strict CSP is untouched.

r/25_build_wpg_address_index.R builds the index from two City of Winnipeg open datasets (Open Government Licence – Winnipeg): the Addresses file (~244k active civic addresses with coordinates and the City's own neighbourhood label) and the Census Boundaries file (the 23 cluster + 12 community-area polygons). Each address point is placed in its cluster and community area by point-in-polygon; the neighbourhood is the City's per-address label. Verified over all 244,175 addresses: every neighbourhood falls wholly inside one cluster, and only 10 addresses miss every polygon.

A census dissemination-area join was tried first and measured worse — DAs are coarser than neighbourhoods in newly-built areas and the lookup assigns each DA wholly to one neighbourhood, so it disagreed with the City's own labels for 11.7% of addresses at neighbourhood level and 4.1% at cluster level. The script header records the detail.

The output is collapsed to per-street, per-parity number runs — 4,556 streets, 6,889 runs, 121 KB (~43 KB gzipped) — and the build replays every address through the written file, aborting if a single one resolves wrongly. Note that 26 of the City's 237 neighbourhoods are newer or finer than the 2021 DA vintage r/12 builds from (Prairie Pointe, Crestview, Leila North, Bridgwater Lakes, Exchange District, …); those addresses still resolve, and the neighbourhood row shows "no profile" while the cluster and community area load normally.

It runs on the monthly refresh. The City's boundaries never move; only the address list does. So the neighbourhood → cluster → community-area hierarchy is derived once from the polygons and committed as r/lib/wpg_nbhd_hierarchy.csv (237 rows), and the routine rebuild is pure text — fresh addresses joined to that CSV by name, needing only jsonlite. That matters because sf (and its GDAL/GEOS system libraries) is not installed on the refresh runner; the fast path never touches it, so the index stays current with new subdivisions automatically.

npm --prefix web run data:wpgaddr                    # fast path (what CI runs)

# Re-derive the hierarchy from the City's polygons — needs sf. Normally you
# never run this by hand: rebuild-wpg-hierarchy.yml does it quarterly.
$env:WPG_ADDR_REBUILD_HIERARCHY="1"; npm --prefix web run data:wpgaddr

The hierarchy rebuilds itself quarterly. rebuild-wpg-hierarchy.yml runs on the 5th of Jan/Apr/Jul/Oct: it apt-installs GDAL/GEOS/PROJ (the only job in the repo that needs them), re-derives the hierarchy by point-in-polygon over all ~244k address points, rebuilds the index, and commits whatever moved. It distinguishes the two: a changed hierarchy means the City moved its geography, so it opens an issue naming the added and removed neighbourhoods; a changed index alone just means new addresses in existing neighbourhoods, which the monthly refresh does all year, so it commits with a data: refresh Winnipeg address index subject and no issue. Either way the commit message says which it was. A failure there breaks nothing: the committed hierarchy and index stay put and the monthly refresh keeps using them, so the only cost is that a brand-new neighbourhood stays unrecognised until the next successful run. That is also why the monthly "new neighbourhood name" alert is a heads-up rather than a task — it resolves itself at the next quarterly rebuild.

Both paths produce a byte-identical index. Three guards keep an automated run from publishing junk: a new neighbourhood name the hierarchy doesn't cover is excluded and written to data/wpg_address_new_nbhds.txt, which the workflow turns into a GitHub issue; the index may not shrink more than 2% in street count against the committed one (override with WPG_ADDR_ALLOW_SHRINK=1); and every address is replayed through the written file before the build ends. The step is best-effort in the workflow, like r/24 — a City open-data outage leaves the committed index in place rather than failing the whole refresh.

Refresh schedule

Workflow	Cron (UTC)	What it pulls
refresh-data.yml	2nd of every month + 28th of Jan + 28th of Jul	Full pipeline: CMHC Rms/Srms/Scss + BoC + StatsCan + Winnipeg address index
refresh-indicators.yml	Every Monday	BoC + StatsCan only (skips the slow CMHC scrape)
rebuild-wpg-hierarchy.yml	5th of Jan / Apr / Jul / Oct	Re-derives the Winnipeg neighbourhood → cluster → community-area hierarchy from the City's polygons (the only job needing sf)

CMHC publishes the Rental Market Survey twice a year (April + October). The Jan/Jul crons are timed to catch the typical release window so rental data refreshes within hours of CMHC publishing.

Email notifications (optional)

Both workflows send an email on failure. The CMHC refresh emails on success only when data actually changed; the indicators refresh emails every run, with a "no new data" subject when nothing was committed, so a missing Monday email means the run did not happen. To enable, add these to the GitHub repo:

• Secret MAIL_USERNAME — sender SMTP username (e.g. Gmail address)
• Secret MAIL_PASSWORD — sender SMTP app password
• Variable MAIL_TO — recipient address
• Optional variables SMTP_HOST / SMTP_PORT — defaults to smtp.gmail.com:587

If any of those is missing the notification step is silently skipped — the workflow itself still runs and (on failure) opens a GitHub issue.

The indicators success email carries a summary built by scripts/refresh_summary.py (also written to the run's Summary tab): which indicator groups advanced and by how many records, the CREA HPI and WRREB as-of months with their stale flags, and any pipeline lines that signal a scrape fell back to last-good data. The Commit link is the data commit the run pushed. The same summary goes out under a "no new data" subject when nothing changed, since a scrape that keeps falling back to last-good data also looks like "no change". That includes r/14's history-shrink guard: a series that comes back with more than 20% fewer records than the last build (a partial StatsCan reply) keeps its last-good history, with this run's values laid over it, and gets a "history shrank" note. Set INDICATORS_ALLOW_SHRINK=1 for a run that shortens a series on purpose without changing its vector. Both schedules run at 04:41 UTC; the earlier 07:23 UTC slot was queued 2–8 hours late by GitHub every time.

Verifying the pipeline

After data:all, run Rscript r/99_verify_samples.R — it makes 5 fresh get_cmhc() calls and compares them to the generated JSON shards. Non-zero exit on mismatch.

Data sources & attribution

All data is from public Canadian sources. Attribution is required by the respective publishers' open-data terms and is surfaced in every Excel export's Metadata sheet, in the per-tab and sitewide footers in the app, plus the in-app indicator chart captions.

• CMHC Rental Market Survey (Rms), Secondary Rental Market Survey (Srms), and Starts & Completions Survey (Scss) via von Bergmann, J. (2025) cmhc: R package to access, retrieve, and work with CMHC data. Use or reproduction of any CMHC data shown here is subject to the CMHC Licence Agreement for the Use of Data, including its attribution requirements. This product is not affiliated with or endorsed by CMHC.
• Statistics Canada Web Data Service via the cansim R package. Contains information licensed under the Statistics Canada Open Licence.
• Bank of Canada Valet API — terms. Free reuse with attribution.

Licence

Code is MIT-licensed — see LICENSE. Bundled data is subject to the source publishers' terms listed above.