Drought & El Niño Monitor

A Streamlit dashboard that tracks rainfall, soil moisture, evapotranspiration, and a calibrated drought-alert trigger at the ADM1 (department / departamento) level. Two countries are wired up today — El Salvador (14 departamentos) and Haiti (10 départements) — from a single codebase with country selection at startup via the COUNTRY env var.

Indicators (refresh cadence in parentheses):

Source	Variable	Cadence
CHIRPS v3 (UCSB)	Pentadal rainfall + SPI-1/3/6	3 days (GEE) + UCSB CHIRPS-Prelim fill for the last few weeks
NOAA GFS 0.25°	15-day rainfall forecast	daily
SMAP L4 (NASA)	Root-zone soil moisture (0–100 cm)	3 days
FAO WAPOR v3	L1 AETI evapotranspiration (~300 m, dekadal)	10 days
IMERG-Late V07 (NASA)	Daily rainfall (event scale)	daily
NOAA ONI	ENSO index for analog-year overlay	monthly

Architecture: one repo, one Docker image, two Cloud Run services (one per country) each backed by its own GCS bucket. A separate static landing page (landing/) lets users pick a country.

Design plan: ~/.claude/plans/i-need-to-design-parsed-hare.md. Background on El Niño impacts: el_nino_agricultural_risks.md.

Live deployments

	URL
🏠 Landing (pick a country)	https://drought.pygis.io
🌾 El Salvador	https://es.drought.pygis.io
🌾 Haiti	https://ht.drought.pygis.io

All three are public (no sign-in), served from Cloud Run in haiti-fews-mmann1123 with Google-managed TLS certs, fronted by custom domains under pygis.io. The original run.app URLs (*-33790149295.us-central1.run.app) still work as fallbacks. ETL refreshes run via Cloud Scheduler daily, starting 05:00 in each country's local timezone (see How data refreshes). The dashboard is read-only — it never triggers a fetch.

---

Local quickstart

Prerequisites

pip install -r requirements.txt
# or, in the existing conda env:
conda activate haiti

For real data you also need a Google Earth Engine project — see GEE auth below.

Option A — synthetic data (no GEE, no auth)

Fastest path to see the UI working. Generates ~30 years of plausible per-departamento series with El Niño dips baked in for historical analog years.

# (from the repo root: /home/mmann1123/Documents/github/)
python -m el_nino.etl.run_etl synth --start 1995-01-01

# Launch with default country (El Salvador)
streamlit run el_nino/dashboard/app.py

# ...or Haiti
COUNTRY=haiti streamlit run el_nino/dashboard/app.py

Open http://localhost:8501.

The synthetic seasonal cycle is ES-tuned (primera/postrera, canícula). Haiti still renders cleanly with synth but the phenology is approximate; use Option B for accurate Haiti data.

Option B — real data via Earth Engine

El Salvador (defaults):

# One-time: bootstrap the AOI polygons from FAO/GAUL
python -m el_nino.etl.aoi.fetch_aoi

# Full historical backfill from 1981 (CHIRPS) — ~15 min
python -m el_nino.etl.run_etl backfill --indicator chirps --start 1981-01-01
python -m el_nino.etl.run_etl backfill --indicator smap   --start 2015-04-01
python -m el_nino.etl.run_etl backfill --indicator wapor  --start 1981-01-01
python -m el_nino.etl.run_etl backfill --indicator imerg  --start 2000-06-01

# Fill the last few weeks (CHIRPS V3 has ~28-day latency; UCSB CHIRPS-Prelim
# is ~3-day latency) and pull the 15-day GFS forecast
python -m el_nino.etl.run_etl prelim
python -m el_nino.etl.run_etl forecast

# Recompute climatology, anomaly z-scores, and freshness
python -m el_nino.etl.run_etl finalize

streamlit run el_nino/dashboard/app.py

Haiti: same commands, prefix everything with COUNTRY=haiti:

COUNTRY=haiti python -m el_nino.etl.aoi.fetch_aoi
COUNTRY=haiti python -m el_nino.etl.run_etl backfill --indicator chirps --start 1981-01-01
COUNTRY=haiti python -m el_nino.etl.run_etl backfill --indicator smap   --start 2015-04-01
COUNTRY=haiti python -m el_nino.etl.run_etl backfill --indicator wapor  --start 1981-01-01
COUNTRY=haiti python -m el_nino.etl.run_etl backfill --indicator imerg  --start 2000-06-01
COUNTRY=haiti python -m el_nino.etl.run_etl prelim
COUNTRY=haiti python -m el_nino.etl.run_etl forecast
COUNTRY=haiti python -m el_nino.etl.run_etl finalize
COUNTRY=haiti streamlit run el_nino/dashboard/app.py

Both countries can share the same local data/ directory — country filtering is applied at read time (see config.country_departments()) so the dashboards never cross-pollute.

GEE auth

earthengine authenticate     # one-time, interactive
echo "GEE_PROJECT=your-gee-project" >> .env

In Cloud Run, mount a service-account JSON via Secret Manager and set GEE_SERVICE_ACCOUNT_JSON instead.

---

Country switching

Everything country-specific lives in config.py under the COUNTRIES registry. The active country's entry is exposed as config.CC and drives:

• AOI geojson filename → departamentos_es.geojson / departamentos_ht.geojson
• Map center + zoom
• Native ADM1 term (departamento / département) used in UI labels and tooltips
• Priority departments (the drought-vulnerable focal set highlighted in the UI)
• Silking / growth window for the calibrated trigger
• Labeled historical drought events for calibration scoring
• Default dashboard language (es for El Salvador, fr for Haiti). The sidebar's ES | EN / FR | EN toggle switches the whole UI to English; the choice is mirrored to ?lang= so links are shareable. All display strings live in dashboard/i18n.py.

Adding a third country = append one entry (including default_lang plus crop_caption_<key> / priority_label_<key> strings in dashboard/i18n.py), run COUNTRY=<key> python -m el_nino.etl.aoi.fetch_aoi, and the same code paths handle it.

---

How data refreshes

In production (Cloud Run + Cloud Scheduler):

Job	Cron (local)	Purpose
${CC}-prelim	05:00 daily	Fill the GEE → today gap with UCSB CHIRPS-Prelim
${CC}-forecast	05:15 daily	Pull the latest 15-day GFS rainfall forecast
${CC}-fetch-chirps	05:30 every 3 days	CHIRPS V3 pentad refresh
${CC}-fetch-smap	05:45 every 3 days	SMAP L4 root-zone soil moisture
${CC}-fetch-wapor	06:00 every 3 days	FAO WAPOR ETa
${CC}-fetch-imerg	06:15 daily	IMERG-Late rainfall
${CC}-enso	06:30 Tuesdays	NOAA CPC ONI + weekly Niño 3.4 SST anomaly
${CC}-finalize	06:45 daily	Recompute SPI + climatology + anomaly z + freshness

Crons run in each country's local timezone (America/El_Salvador for ES, America/Port-au-Prince for HT), so the chain starts at 05:00 local year-round. Haiti observes DST and El Salvador does not, so a UTC-pinned cron would drift an hour twice a year for HT only.

Entries are spaced 15 min apart so they don't race the same Cloud Run slot — individual runs take 2–9 minutes and write to the same parquets. finalize runs last because it derives everything from whatever the fetches above just landed. ${CC} is es or ht — 8 entries per country, 16 total. See deploy/schedule.sh, which is idempotent (delete-then-create) and safe to re-run.

Manually, for an off-schedule catch-up:

python -m el_nino.etl.run_etl prelim && python -m el_nino.etl.run_etl forecast

The dashboard itself never fetches. It used to offer a "🔄 Check for new data" sidebar button, which was removed because running an ETL pull inside the serving process pins a Cloud Run instance for the duration of the pull — and the scheduler already does the same work daily.

---

Trigger calibration

Alerts fire when SPI-3 < threshold AND SMAP root-zone anomaly < threshold in any one of the country's departments during the season's alert window. Each country has two growing seasons; an alert stays on for the rest of the season it fires in and resets when the next season starts. Operating points are calibrated per-country against labeled historical drought events:

	El Salvador	Haiti
Window	DOY 196-227 (mid-Jul to mid-Aug)	DOY 152-227 (Jun 1 to Aug 15)
SPI-3 threshold	< −1.5	< −1.3
SMAP RZSM threshold	< −1.0σ	< −0.5σ
Recall	100% (2015, 2018, 2023, 2026)	67% (2015, 2018)
Severe recall	100% (catches 2015)	100% (catches 2015)
Precision	100%	40%
FP / decade	0.0	2.7

Full reports:

• experiments/trigger_calibration_report_es.md
• experiments/trigger_calibration_report_ht.md

Re-run after each annual data refresh:

python -m el_nino.experiments.trigger_calibration                # ES
COUNTRY=haiti python -m el_nino.experiments.trigger_calibration  # HT

---

Deploy to Google Cloud

Three scripted, idempotent steps per country plus a one-time image build. See deploy/README.md for the full guide; the short version:

# El Salvador (defaults)
PROJECT=haiti-fews-mmann1123 bash deploy/setup_infra.sh
gcloud builds submit --config deploy/cloudbuild.yaml .
PROJECT=haiti-fews-mmann1123 bash deploy/deploy_job.sh
PROJECT=haiti-fews-mmann1123 bash deploy/schedule.sh

# Haiti (two extra env vars per call)
COUNTRY=haiti COUNTRY_CODE=ht PROJECT=haiti-fews-mmann1123 bash deploy/setup_infra.sh
COUNTRY=haiti COUNTRY_CODE=ht PROJECT=haiti-fews-mmann1123 bash deploy/deploy_job.sh
COUNTRY=haiti COUNTRY_CODE=ht PROJECT=haiti-fews-mmann1123 bash deploy/schedule.sh

Each country gets its own bucket, service account, Cloud Run Job, and six Scheduler entries. The shared Docker image is built once.

Analytics: all three public surfaces report to one Google Analytics 4 property (G-N16M48WHG1), committed as the deploy default and split by GA4's Hostname dimension. Local runs serve no tag. See deploy/README.md.

Landing page: a static HTML page with a video background lives in landing/ — two country cards that link to each country's Cloud Run URL. Deploy as a third tiny Cloud Run service (Dockerfile pending, see the landing folder for the static asset and the design).

---

Layout

el_nino/
├── config.py                 # COUNTRY registry, paths, baselines
├── README.md                 # this file
├── el_nino_agricultural_risks.md   # background on ES/HT El Niño impacts
├── notes.md                  # ES agronomy notes
├── etl/
│   ├── run_etl.py            # CLI: synth | fetch | backfill | prelim | forecast | finalize | climatology | enso
│   ├── refresh_check.py      # manual catch-up orchestrator (GEE + prelim + forecast) + freshness writer
│   ├── chirps_prelim.py      # UCSB CHIRPS-Prelim daily-TIFF gap fill
│   ├── triggers.py           # CalibratedTrigger runtime evaluator (country-aware)
│   ├── synth.py              # synthetic data for local dev
│   ├── storage.py / climatology.py / enso.py / freshness.py / gee.py
│   ├── aoi/
│   │   ├── fetch_aoi.py      # one-time FAO/GAUL bootstrap per country
│   │   ├── departamentos_es.geojson
│   │   └── departamentos_ht.geojson
│   └── indicators/           # chirps.py / smap.py / wapor.py / imerg.py
├── dashboard/
│   ├── app.py                # Streamlit entry (Overview / Indicator Detail / Year Compare)
│   ├── alerts.py             # drought-alert banner + confidence panel
│   ├── map.py                # departamento choropleth (CARTO Positron)
│   ├── charts.py             # climatology envelope figures
│   ├── data.py               # parquet read layer (country-filtered)
│   ├── drought_status.py     # USDM classification + plain-language labels
│   ├── freshness.py          # data-freshness badges
│   ├── status.py             # current-status helpers
│   ├── auth.py               # Streamlit OIDC gate (disabled by default — public)
│   └── site_footer.py        # GWU mark + data attribution + disclaimer
├── experiments/
│   ├── trigger_calibration.py
│   ├── trigger_calibration_report_es.md
│   └── trigger_calibration_report_ht.md
├── deploy/                   # Cloud Build + Cloud Run + Cloud Scheduler scripts (see deploy/README.md)
├── landing/
│   └── index.html            # static landing page (video bg, two country cards)
├── static/                   # GW_GE.png logo + landing video
├── data/                     # local-only, gitignored
├── Dockerfile
├── requirements.txt
└── .env.example

---

Attribution

Created by Michael Mann, PhD — George Washington University, Department of Geography & Environment, Columbian College of Arts & Sciences. Independent dashboard; not affiliated with or endorsed by the data providers listed above. Indicators and forecasts are provided without warranty of accuracy or fitness for any purpose.