Critical raw materials — mine ▸ refinery ▸ trade atlas

Live → https://criticalmaterialsatlas.org/

A public-data atlas of critical raw materials: for each one, where it is mined, where it is refined, and the real global trade between every country pair — both directions, for any country you pick. Built to make one thing visible that most trade dashboards hide: the refiner is usually not the source. An apparent China dependence on cobalt is, upstream, a Congo mining dependence funnelled through a Chinese refinery — China is the chokepoint, not the mine.

It's a set of static HTML pages (vanilla JS, no framework) generated by a Python build pipeline and served from GitHub Pages, plus published JSON open-data in out/. No backend.

Headline finding → FINDINGS.md: across critical materials, the top exporter is not the top miner in 17 of 32 of them — import-origin statistics systematically overstate how diversified supply really is. Reproduce with reconcile/findings.py. Companion → FINDINGS-2.md: re-export theatre (⅔ of platinum/palladium flows route through London/Zurich) and mirror gaps (51% of two-sided flows disagree >2×) — why raw trade needs reconciling.

What it shows

Four layers per material, from public sources:

Layer	Source	What it is
Reserves / potential	USGS Mineral Commodity Summaries (approx.)	world reserve shares — where it could come from, exploited or not
Mined	USGS Mineral Commodity Summaries (approx.)	world mine-production shares
Refined	best per material: BGS World Mineral Statistics (annual actuals, 7 metals — copper, cobalt, nickel, alumina, magnesium, germanium, arsenic), else IEA 2026 / EU CRM 2023 / USGS	where ore becomes usable metal — usually the real chokepoint
Traded	UN Comtrade, via CEPII BACI HS17 V202601, 2018–2024 (year-selectable)	complete reconciled bilateral trade — ~16k flows/yr, ~210 countries
Traded — 2025*	raw UN Comtrade, self-reconciled (reconcile/), provisional	a nowcast: our BACI-validated reconciliation of partial 2025 data — indicative only
Traded — 2026**	Q1 monthly Comtrade momentum + Pink Sheet prices, directional	2025 structure scaled by per-material momentum — shares = 2025, only levels tilt; a trend signal

Four views over the same data: Flow (Sankey), Map (choropleth + curved trade arrows), Globe (animated arcs), Table (each material mined ▸ refined ▸ top exporter · top importer, with a "country domination" sort that arranges rows into an approximate triangular matrix to dramatise concentration). Arrows/arcs are coloured by product stage: amber = raw ore, violet = refined product.

The European Union is available as one lens among others (per-material EU import-origin correction); it is no longer the frame. See MATERIALS.md for the full material → HS-code → stage crosswalk and the headline figure from each layer.

How country roles are derived

Country fills on the map/globe are derived from the trade data, not a hardcoded top-N list:

• a miner (amber) = a country that actually exports the raw ore (for raw-ore materials);
• a refiner (violet) = a country that actually exports the refined product (for refined materials);
• where the relevant form isn't in that trade stream (e.g. an ore's refined form sits under a different HS code), it falls back to the USGS/IEA reference shares.

Caveat: customs trade can't separate a true refiner from a re-export hub — Hong Kong, Singapore, the Netherlands can surface as "refiners" of material they merely trans-ship. The role colours are trade exposure, not proof of physical processing. The full set of caveats (mixing trade with approximate reference shares, value-vs-tonnage, structural Sankey links, sqrt colouring) is in methodology.html → Limitations.

Build pipeline

# 1. Global bilateral trade → out/flows_2018.json … flows_2024.json
#    Streams the CEPII BACI HS17 yearly CSVs (downloaded into raw/baci/, gitignored), filters to the 32
#    HS6 codes, maps BACI country codes → ISO2, keeps top suppliers/customers per country. BOM-free JSON.
powershell -File build_flows_years.ps1

# 2. Mine/refine reference layers + the EU Comext lens → out/data.json + per-material PNGs
Rscript build_static.R

index.html reads out/data.json (mine/refine shares + EU lens) and out/flows_<year>.json (global trade, picked by the year slider) — no code change to add a material once its data is present.

Analytical suite

Beyond the interactive map, the repo generates ~40 standalone analysis pages from ~35 build_*.py scripts — each emits a JSON artifact and a self-contained HTML page. Grouped roughly as the site nav does:

• Supply risk & criticality — build_risk.py, build_criticality.py, build_riskmethods.py, build_risk_adjusted.py (transparent multi-component supply-risk index, WGI governance-weighted)
• Supply structure — build_refining.py (ore-vs-refined wedge), build_network.py / build_network_sensitivity.py (chokepoint / PageRank), build_companionality.py, build_complexity.py, build_origin.py
• Demand & scenarios — build_demand.py, build_net_demand.py, build_scenarios.py, build_mining_expansion.py, build_recycling.py, build_trends.py
• Rigor & cross-checks — build_satellite.py (mine-footprint), build_robustness.py, build_uncertainty.py, build_host_shock.py, build_synthesis.py

The scripts are order-dependent (e.g. fetch_wgi.py → build_risk.py → build_criticality.py; insights last). CADENCE.md is the canonical end-to-end runbook; refresh.ps1 drives only the R / Eurostat-Comext half. A legacy app.R Shiny dashboard is an optional interactive counterpart, not part of the published static site.

Layout

• index.html — the interactive tool (Flow / Map / Globe / Table); repo root = the GitHub Pages site
• out/data.json — mine (USGS) + refine (IEA) shares + EU Comext import-origin lens
• out/flows_2018.json … flows_2024.json — complete BACI bilateral trade, one file per year (committed)
• build_flows_years.ps1 — BACI HS17 yearly CSVs → out/flows_<year>.json (multi-year, one nomenclature)
• reconcile/ — Python nowcast engine, consolidated into this repo from the former standalone comtrade-reconcile (now archived; its reproducibility CI runs here via .github/workflows/engine-validate.yml, and it ships committed public-data fixtures + a pre-registration). BACI-style reconciliation of raw Comtrade for years BACI lacks — see reconcile/README.md for the full method note + validation: pull_comtrade.py (raw bilateral pull) → reconcile.py (CIF/FOB + reliability weights + mirror averaging) → validate.py (vs official BACI 2024: top-1 exporter 25/30, share MAE 3.5%, HHI corr 0.92) → build_recon_flows.py (→ out/flows_2025.json) → build_2026_nowcast.py (→ out/flows_2026.json, directional)
• build_baci_flows.ps1 — single-year HS22 builder (legacy; superseded by the multi-year script)
• build_static.R — regenerates out/data.json + the per-material PNGs in out/
• methodology.html — one-page method note: the three layers, the refiner-vs-source correction, the EU Comext origin correction, and the Limitations
• MATERIALS.md — material → HS6 → stage crosswalk with headline figures per layer
• VALIDATION.md — spot-checks of each layer against published USGS/IEA figures
• raw/ — downloaded source data (BACI zip/CSV, Comext), gitignored

Provenance & scope

Independent demonstration from public data — USGS Mineral Commodity Summaries, BGS World Mineral Statistics (OGC API), IEA Critical Minerals, EU CRM 2023 (JRC/SCRREEN2), UN Comtrade (CEPII BACI), Eurostat Comext. Not affiliated with, nor representing, any institution. Figures are approximate and rounded; read each row as an overlay of different measures, not one observed supply chain.

Each material profile carries a per-layer year slider where the source publishes an annual series: Mined 2019–2023 (USGS editions), Reserves 2019–2023 (USGS), Refined 2019–2024 (BGS), and Traded 2018–2026 (reconciled BACI + nowcast) — so you can watch a stage's country mix shift over time, not just a snapshot. build_bgs_refined.py pulls the refined series from the free BGS API (no key).