Impacto Acumulado
Cumulative environmental impact of renewable energy projects in Andalusia,
built from the official gazettes (BOE and BOJA).
Every solar or wind park in Spain is assessed on its own. This project turns
the environmental declarations and authorisations published in the gazettes
into structured data and shows the sum per municipality, per protected area
and per province, over time.
Status: pipeline implemented (fetch, extract, resolve, aggregate, export,
reference loads, evaluation harness, CI and weekly workflow). Production
database live on Neon with the reference layers loaded; the weekly workflow
runs against it and commits exports. Backfill: the BOE range 2019-01 to
2026-09 is swept and audited, the BOJA range has no documents for 2020 or
2021. 580+ projects resolved from 700+ BOE and BOJA documents
(2019 onward; the count grows with each weekly export), extracted with Mistral ministral-14b-latest under
prompt v4 (re-extracted 2026-09-28; see the extraction-fixes spec). See
"Production setup" below. Web site: map, municipality and project pages, methodology with measured accuracy and a data download page at https://impacto-acumulado.vercel.app (slices 1 to 3). Slice 3 adds a technology filter, Natura 2000 and sensitivity overlays, a province table and a monthly timeline of decisions to the map page.
Pipeline
Python package in pipeline/. Stages: fetch (BOE and BOJA), extract (LLM to typed JSON), resolve (documents to projects), aggregate (PostGIS), export (CSV and GeoJSON into web/public/data/).
docker compose up -d db
cd pipeline && uv sync
uv run python -m impacto migrate
uv run python -m impacto fetch --from 2024-01-01 --to 2024-03-31
uv run python -m impacto extract --limit 20
uv run python -m impacto resolve && uv run python -m impacto aggregate && uv run python -m impacto export
uv run pytest
Configuration is through environment variables; see pipeline/env.example. Sources, filters and reference layers are documented in docs/sources.md. Extraction accuracy is measured with uv run python -m impacto eval (harness and labels in pipeline/evaluation/).
Web
Next.js app in web/, fully static: pages read web/public/data/ at build time and Vercel rebuilds on every commit to main, including the weekly data commit. No database connection from the web.
cd web && npm ci
npm run dev # http://localhost:3000
npm test # vitest, loaders and pure helpers against tests/fixtures/data
npm run build && npm run e2e # playwright and axe against the built site
Routes: /, /municipio/[ine], /proyecto/[id], /metodologia, /datos, /sitemap.xml. The dataset is published under CC BY 4.0 on /datos.
Vercel project impacto-acumulado (team lauramiros-projects): root directory web, framework Next.js, Node 24, no environment variables. Production: https://impacto-acumulado.vercel.app (first deploy 2026-09-21 from d2f355f with vercel --prod). Connected to the GitHub repo: every push to main deploys to production; vercel --prod from the repo root also works.
Lighthouse 13.5 (mobile, Chrome headless, 2026-09-21): / performance 88, accessibility 100 (LCP 3.3 s; the map loads a 196 KB compressed municipalities_map.geojson simplified at 200 m, the first deploy scored 73 with the 531 KB full-fidelity file); /municipio/11020 performance 99, accessibility 100 (LCP 2.0 s); /proyecto/1 performance 99, accessibility 100 (2026-09-22).
Changes to the published data
Also listed under "Cambios" on /datos; add new entries to both.
-
2026-10-06: display only, data files unchanged: municipality, developer and /promotores figures no longer count projects with no verdict in the bulletin (desconocido) as "aprobados o en trámite", the headline's basis; they are given on a row, line or column of their own. Project lists show the peak MW ("13,6 MW pico") when a project has no nominal MW.
-
2026-10-06: BOE links open the readable page: url in documents.csv, document_urls in projects.csv, "Ver en el BOE" and the feed entries pointed at the document XML (https://www.boe.es/diario_boe/xml.php?id=<id>, served as application/xml); they now point at https://www.boe.es/diario_boe/txt.php?id=<id>, the HTML page for the same id. fetch stores the summary's url_html (or builds it from the identifier) and still reads the text from url_xml; migration db/migrations/008_boe_html_urls.sql rewrites the BOE rows already in raw_documents. Applied in the 2026-10-06 rebuild: 192 url values in documents.csv and the document_urls of 105 projects changed, nothing else; BOJA links are unchanged.
-
2026-10-06: duplicate projects: after scoring, two groups that share a developer (impacto.developers key) and a plant name (blocking.plant_key, or one plant of a multi-plant title) join when each is the other's only namesake and nothing tells them apart (a procedure or phase conflict, disjoint municipalities, a line against a plant, wind against solar); match_reason reads namesake. Ambiguous namesakes stay apart and review-clusters lists them under namesake. On the rebuild of 2026-10-06 projects.csv went from 583 to 581 rows: 675 joined 35 (Rey I to IV) and 61 joined 10 (Lirios Solar PV), and the retired ids redirect. Carmona (41024) fell by 356.4 MW, from 1,945.3 to 1,588.9. Two pairs the rule meets on production data stayed apart and wait in the review queue under namesake: 699 with 75 (Saucito) and 698 with 47 (Ronda I to III); decide them with resolution_overrides.
-
2026-10-06: turbines and technology are checked against the project name (impacto/extract/validate.py, check_generation, applied at extraction and again at every resolve): turbines on a project named photovoltaic or a module with no wind of its own are dropped (Retuerta, Valdefuentes, Tallisca, PV Centenar, Ferreira II, Montegordo), and a name led by "eólico" turns solar_fv or into (32 Filabres, Peregiles and La Rambla; 262 Parapanda). No row carries turbines.
Continuous integration and the weekly run
.github/workflows/ci.yml runs ruff check and pytest against a PostGIS service container on every push to main, every pull request, and on manual dispatch..github/workflows/pipeline.yml (weekly-pipeline) runs Mondays at 06:00 UTC and on manual dispatch: migrate, fetch (last 14 days by default), extract, resolve, aggregate, export, then commits web/public/data/ if it changed. Manual inputs: from (fetch start date), extract_limit (default 200) and model (exported as IMPACTO_LLM_MODEL when set). It reads the NEON_DSN and GROQ_KEY repository secrets. An exhausted Groq daily quota stops the extract stage early without recording an attempt against the document, and the step is continue-on-error so resolve, aggregate and export still run on what was extracted.
Production setup
The weekly workflow needs a production database and the repository secrets. Steps 1 to 3 and 5 were done on 2026-09-20 and 2026-09-21 (Neon project calm-sunset-94532458, branch production, Frankfurt; secrets NEON_DSN, GROQ_KEY, MISTRAL_KEY; reference counts match the dev load; the first scheduled run on 2026-09-21 completed in 9 minutes and committed an export). Step 4 is in progress. The steps are kept as the record of what was done and how to redo it against a fresh database.
To run a stage against Neon from your machine without editing pipeline/env.local, export the DSN for the shell only (the Neon CLI is linked to the project through .neon):
cd pipeline
export IMPACTO_DB_DSN="$(neon connection-string --project-id calm-sunset-94532458 --branch production | tr -d '\r\n')"
uv run python -m impacto extract --provider mistral --limit 20
Neon terminates idle connections when the endpoint suspends or restarts, which happens while extract spends minutes in LLM calls; the extract stage reconnects and retries the save once when that happens, so a dropped connection costs nothing.
-
Create the Neon project. In the Neon console create a project named impacto-acumulado in an EU region (Frankfurt). No manual extension setup is needed: the first migrate run executes CREATE EXTENSION postgis, which Neon allows. Copy the direct (non-pooled) connection string, the one whose host does not contain -pooler: the pipeline is a single long-lived connection that runs multi-statement migrations and holds transactions across bulk loads, which PgBouncer's transaction-mode pooling behind the pooled endpoint does not support reliably.
-
Add the repository secrets. In GitHub, Settings, Secrets and variables, Actions, create:
NEON_DSN: the Neon direct connection string.GROQ_KEY: the Groq API key.MISTRAL_KEY (optional): the Mistral API key, used only when the workflow is dispatched with provider set to mistral.
-
Run the migration and the reference loads once against Neon from your machine, with IMPACTO_DB_DSN set to the Neon connection string (put it in the local config file described in pipeline/env.example; never commit it). The reference files and field names are the ones recorded in docs/sources.md ("Task 10 load results"); the sensitivity layer is loaded once per technology, so there are four commands for the three reference subcommands:
cd pipeline
uv run python -m impacto migrate
uv run python -m impacto reference municipalities ../tmp/reference/dera_extracted/13_01_TerminoMunicipal.shp --code cod_mun --name nombre --province provincia
uv run python -m impacto reference protected-areas ../tmp/reference/natura2000_extracted/RedNatura2000/InfGeografica/InfVectorial/Shapes/RedNatura2000_Andalucia.shp --code CODIGOEURO --name NOMBRE --type FIGURA
uv run python -m impacto reference sensitivity ../tmp/reference/eol_extracted/Clas_ISA_eol_pb.tiff --technology eol
uv run python -m impacto reference sensitivity ../tmp/reference/ftv_extracted/Clas_ISA_ftv_pb.tiff --technology ftv
Expected counts (from the dev load): municipalities 785, protected areas 252 read and 197 stored, sensitivity eol 19985 polygons, sensitivity ftv 27179 polygons.
-
Run the backfill from a machine with a suitable LLM budget, not from Actions. The Groq free tier for openai/gpt-oss-120b is capped at 200,000 tokens per day (observed and recorded in docs/sources.md), which is about two to three documents per day, so the backfill from 2019 needs either a paid Groq plan or a different provider budget. With IMPACTO_DB_DSN pointing at Neon and IMPACTO_LLM_KEY set:
