Electronic CPH — Copenhagen electronic music events

A fast, curated, largely self-maintaining index of electronic music events in Copenhagen & Frederiksberg. See the product brief this was built against for full requirements; this file covers what's implemented, how it's organised, and what's intentionally left for later phases.

Stack

Next.js (App Router) + TypeScript + Tailwind CSS v4 + Postgres (Drizzle ORM).

cp .env.example .env.local   # point DATABASE_URL at a local/hosted Postgres
npm install
npm run db:migrate         # apply the schema
npm run db:seed:dev         # local/demo only — real registries + sample events (see "Seeding" below)
npm run dev                  # http://localhost:3000
npm run test                   # vitest — datetime, dedup, normalization, classification, ICS export, sync
npm run build                    # production build (all routes are statically generated except /api/*)
npm run lint
npm run db:verify-sync             # end-to-end proof: live-fetches Hangaren + exercises every sync scenario
npm run db:verify-bootstrap          # end-to-end proof: empty DB -> migrate -> production bootstrap -> real sync

Local database guard — a safety net for local development only. dev, db:migrate, db:studio, db:seed:dev, db:verify-sync and db:verify-bootstrap first check that DATABASE_URL is a local test database (localhost / 127.0.0.1 / ::1 / a Unix socket) and refuse a hosted one (src/db/localDatabaseGuard.ts). An exported DATABASE_URL wins over .env.local, so a shell that still carries a hosted connection string is stopped instead of reaching it. To run one of them against a hosted database on purpose, set ALLOW_HOSTED_DATABASE=<that command> for that one run (e.g. ALLOW_HOSTED_DATABASE=db:migrate npm run db:migrate, as the "Prepare Production Database" workflow's migrate step does) — an opt-in written into an env file is ignored. db:seed:dev never runs against a hosted database. For npm run dev the guard checks every env file Next loads (.env.development.local, .env.local, .env.development, .env). npm run build is not guarded — the Vercel Production build applies migrations itself (src/db/migrateOnDeploy.ts).

The guard does not protect Production workflows and is not their access control. The GitHub Actions workflows and the Vercel build run their scripts directly with the repository's DATABASE_URL secret, outside these six commands: Production writes are gated by workflow_dispatch plus a typed confirmation, and the "read-only" audits/inspections by their own code opening a BEGIN READ ONLY transaction — they currently still use the privileged credential, not a read-only one. npm test never sees a DATABASE_URL or ALLOW_HOSTED_DATABASE (vitest.setup.ts removes both); unit tests mock the database.

Current phase

Phase 1 (visual/product proof), Phase 2 (taxonomy, venue registry, dedup/normalization, source registry, adapter architecture), a real persistent Postgres database with a real admin write path and field-level manual-override protection, and two real, live, working first-party ingestion sources (Hangaren and Culture Box) are all implemented and wired end to end: adapter → extraction → validation → normalization → dedup → genre classification → confidence gate → database → public site → subsequent source update, each running on an actual schedule (.github/workflows/sync-hangaren.yml and .github/workflows/sync-culture-box.yml, every 6h — see "Scheduling" below) rather than only being triggerable by hand. /admin and /api/admin/* are gated behind HTTP Basic Auth (see "Admin access" below) — not publicly writable. src/lib/data/*.ts is no longer read by the running app — it's the seed source for both seed paths (see "Seeding" below) and fixture data for pure-logic unit tests. The other two first-party venues (Gravity, Den Anden Side) are evaluated in src/lib/data/sources.ts's integrationNote per source but do not have a working adapter yet — see that file before adding one.

Seeding

Two deliberately separate entry points — never run the dev one against a real deployment:

• npm run db:seed:dev (src/db/seedDev.ts) — local/demo only. Loads the real venue/source registries plus Phase-1 sample events, sample discovery-queue items, and the sources' staged demo sync-health states (e.g. Gravity's fabricated "degraded" example) — useful for seeing the full UI locally with something in it.
• npm run db:seed:production (src/db/bootstrapProduction.ts) — the only one safe to run against a real database. Seeds only the venue and source registries (real Copenhagen venues, real research about real external sources — genuinely required for the app to operate, e.g. the Hangaren pipeline can't resolve a venue without the v-hangaren row existing) and inserts zero demo content: no sample events, no sample discovery-queue items, and no fabricated source sync-history — each source's health fields start neutral (null/0, "never synced yet") and are only ever written by a real sync run afterward. Both commands are idempotent (upsert by primary key); re-running the production one specifically never resets a source's real accumulated sync history back to neutral — see src/db/referenceData.ts's doc comments. Proven end-to-end (empty database → migrate → this bootstrap → a real live Hangaren sync → a second bootstrap run) by npm run db:verify-bootstrap.

Schema migrations reach Production automatically

npm run build runs src/db/migrateOnDeploy.ts before next build. On a Vercel Production build it applies any pending migration in src/db/migrations/; if that fails for any reason the build fails, so the new code is never promoted and the current deployment keeps serving (and the GitHub Actions syncs, which all call the deployed /api/sync/*, keep running the old code too). Local builds and Vercel Preview builds skip it. Details: src/db/deployMigrations.ts.

• How it knows it's Production: VERCEL/NOW_BUILDER (always set by Vercel's builder) mark a Vercel build; VERCEL_TARGET_ENV, falling back to VERCEL_ENV, says which environment. If a Vercel build ever lacks both, it can't tell Production from Preview, so it applies nothing and only checks, read-only: no pending migrations → build continues; pending migrations → build fails (apply them with the workflow below, then redeploy). Code can't ship ahead of its schema either way.

• All pending migrations run in one transaction behind an advisory lock: a failure applies nothing, and two Production builds at once can't both apply the same migration (the second waits, then finds nothing pending). DDL gives up after a 10s lock wait rather than stalling live queries.

• Only additive statements are applied automatically — CREATE TABLE, ADD COLUMN (nullable, or NOT NULL with a DEFAULT), ADD CONSTRAINT … FOREIGN KEY, non-unique CREATE INDEX. These are the only shapes that can't break the deployment still serving while the build runs (or a Vercel instant rollback). Anything else (drop, rename, type change, SET NOT NULL, data UPDATE) fails the build on purpose. To ship one: first deploy code that no longer depends on the old shape; then merge the migration (its build will fail), apply it with the "Prepare Production Database" workflow (migrations_only), and redeploy.

• Bookkeeping is Drizzle's own drizzle.__drizzle_migrations table, so npm run db:migrate and the manual workflow below stay fully interchangeable with it.

Preparing the production database

.github/workflows/prepare-production-db.yml runs the two commands a real deployment's database needs — npm run db:migrate then npm run db:seed:production — against whatever DATABASE_URL the DATABASE_URL repository secret points at. It is manual-only (workflow_dispatch, no schedule, no trigger on push/PR), never runs db:seed:dev or db:verify-bootstrap (the destructive one — see "Seeding" above), never prints DATABASE_URL, and makes no deployment or infrastructure changes.

1. Add the DATABASE_URL repository secret. In the repo on GitHub: Settings → Secrets and variables → Actions → Secrets tab → "New repository secret". Name: DATABASE_URL. Value: the production Postgres connection string (e.g. your Supabase connection string). Save.
2. Run the workflow manually. Actions tab → "Prepare Production Database" in the left sidebar → "Run workflow" → in the confirm box type exactly PREPARE-PRODUCTION-DB → "Run workflow". It will fail immediately (before touching the database) if DATABASE_URL isn't set or the confirmation text doesn't match exactly.
3. Verify success. Open the completed run: both the "Apply database migrations" and "Seed production reference data" steps should show a green check with no error output. Separately, confirm against the database itself (e.g. via psql or Supabase's table editor) that the venues and sources tables are populated and that events/discovery_queue are empty — exactly what npm run db:seed:production guarantees (see "Seeding" above).

Architecture

src/lib/
  types.ts            Canonical data model (Event, Venue, Source, DiscoveryQueueItem…)
  taxonomy.ts          Controlled subgenre taxonomy + homepage display-label logic
  datetime.ts           Nightlife-aware date logic (Europe/Copenhagen, 06:00 cutoff) — see below
  format.ts             Display formatting built on datetime.ts
  normalize.ts           Venue alias resolution + artist name normalization
  dedup.ts                Fuzzy duplicate detection across sources
  classification.ts        Genre evidence hierarchy, quality gate, canonical source priority
  search.ts                 Free-text search over title/artists/venue/subgenre
  links.ts                   External link dedup + CTA priority
  ics.ts                      Calendar export (ICS + Google + Outlook), DST-safe
  jsonld.ts                    schema.org MusicEvent structured data
  queries.ts                    Data-access layer, backed by Postgres (src/db)
  sourceHealth.ts                Source registry health classification
  sync.ts                         Pure sync-time merge decisions: does a candidate match an
                                   already-known event (linked/fuzzy), and if so what actually
                                   changed (buildSyncPatch — dateChanged/timeChanged flags, etc.)
  data/                            Seed fixtures shared by both seed paths + pure-logic unit tests
  sourceRegistry.ts                Pure logic: fixture -> production-safe source row (strips
                                    fabricated demo sync-health, see "Seeding" below)
  adapters/
    types.ts                        RawCandidateEvent — the shape every adapter must produce
    firstPartyAdapter.ts              Generic reference adapter for a first-party JSON feed
    hangarenAdapter.ts                 Real, working adapter — live HTML fetch + parse (see below)
    deterministicGenreMapping.ts       Keyword fallback — evidence tier 5, never tier 1
    pipeline.ts                         EXTRACTION → VALIDATION → NORMALIZATION → DEDUP →
                                         CLASSIFICATION → CONFIDENCE → PUBLISH/REVIEW, as pure
                                         functions. Adapters only ever produce RawCandidateEvent;
                                         everything downstream is adapter-agnostic.
src/db/
  schema.ts / client.ts / mappers.ts  Drizzle schema, pooled connection, DB-row <-> app-type mapping
  writes.ts                            All admin/sync writes — the enforcement point for override
                                        protection (applySourceSyncPatch strips overridden fields)
  sync.ts                               runSourceSync(): orchestrates one full sync run for one
                                         source — fetch, pipeline, match-or-create, write, with
                                         source-failure and zero-events handled as distinct,
                                         never-silent, never-a-cancellation outcomes
  verifySync.ts                         `npm run db:verify-sync` — end-to-end proof against a real
                                         Postgres database (see "Hangaren ingestion" below)
  referenceData.ts                      seedVenues() + seedSourcesProduction() — shared by both
                                         seed paths (see "Seeding" below)
  seedDev.ts                            `npm run db:seed:dev` — local/demo only
  bootstrapProduction.ts                `npm run db:seed:production` — the production-safe one
  verifyProductionBootstrap.ts          `npm run db:verify-bootstrap` — proves empty DB -> migrate
                                         -> production bootstrap -> real sync end to end
src/app/api/sync/[source]/route.ts      Scheduling entry point, `x-sync-token`-protected
src/proxy.ts                             HTTP Basic Auth gate for /admin + /api/admin/*
src/db/inspectSource.ts                  Permanent read-only source diagnostics (see
                                          SOURCE_ONBOARDING.md), wrapped by inspect-source.yml
scripts/scaffold-source.mjs              New-source adapter/test/workflow scaffold generator
.github/workflows/sync-hangaren.yml      Actual cron trigger — see "Scheduling" below
.github/workflows/inspect-source.yml     Permanent read-only source diagnostic workflow
.github/workflows/validate-source.yml    Permanent source-branch validation workflow
SOURCE_ONBOARDING.md                     The end-to-end process for adding a new source

Hangaren ingestion (the one real, live source)

src/lib/adapters/hangarenAdapter.ts fetches https://www.hangaren.dk/events for real — not a fixture — and parses the server-rendered HTML. It was chosen over Culture Box, Gravity and Den Anden Side after checking all four for robots.txt permission, structured data, reliability and maintenance burden (see the integrationNote on each in src/lib/data/sources.ts): Hangaren's /events page lists every upcoming event on one request with semantic <time datetime> tags and a Google Calendar link carrying exact UTC start/end instants, and its robots.txt disallows only the ?format=json/?format=ical shortcuts (named crawlers included), never the plain page.

No explicit genre field exists on the source, so genre classification falls back to a keyword match — but the evidence hierarchy (classification.ts) already distinguishes a keyword match against the source's own description text about this specific event ("official-description", high confidence) from a generic title-only guess ("deterministic-mapping", medium). Hangaren's bios routinely state the genre outright ("Hard Bounce, Schranz and Techno are genres that define the sound of Kander"), so the adapter runs the keyword match against the full bio and credits it at the correct (high) tier instead of the generic fallback — this isn't a weaker gate, it's crediting real evidence the rules already call for. On a real live fetch this resolves 10/19 events to high confidence (auto-published) and leaves 9/19 in review, all for the same single reason: the bio genuinely never states a genre, not a missing date/venue/title/URL. See "Manual review workload" below for what that means week to week. Updates to already-known events (date/time/lineup changes) apply automatically and respect manual overrides regardless. Run npm run db:verify-sync for a live, repeatable proof of the whole flow.

Manual review workload

On a real live fetch (19 upcoming events currently on the page, spanning roughly 13 weeks — ~1.5 events/week): 10 auto-publish, 9 need one review-queue action each (confirm genre, click publish) — but that 9 is a one-time backlog from the first sync ever, not a per-week number. Once caught up, a normal week only surfaces new candidates never seen before (~1.5/week at Hangaren's posting cadence), of which historically about half clear the high-confidence bar automatically; the rest are a single confirm-and-publish click each. Changes to events already tracked (a lineup addition, a time slip) apply with zero admin action, protected fields survive untouched. Estimated steady-state workload: well under one manual action per week for this source — consistent with the brief's "a few minutes of exception handling," not "no review ever."

Nightlife date logic (datetime.ts)

Club nights routinely run past midnight, so a plain calendar day breaks "Tonight", weekend filters and archival timing. Every instant maps to a nightlife day: its calendar date, except anything before 06:00 local still belongs to the previous night. isTonight, isThisWeekend, isNextWeekend, month grouping, and archival (isPastEvent) all key off this, not the raw calendar date. now is read from the visitor's own browser clock (EventExplorer mounts before computing it) rather than baked in at build/request time, so correctness doesn't depend on cache freshness. See src/lib/datetime.test.ts for the edge cases this covers (midnight/month/year rollovers, weekend boundaries, overnight archival).

Source roles and integration status (src/lib/data/sources.ts)

Every external reference is classified independently by role (discovery / ingestion / verification / link) — the strongest discovery source isn't automatically the strongest ingestion source. All four first-party venue/promoter pages (Culture Box, Hangaren, Den Anden Side, Gravity) are marked autoPublish: true in the data model, but only Hangaren and Culture Box currently have a real, working adapter wired to the scheduler (src/lib/adapters/hangarenAdapter.ts / POST /api/sync/hangaren and src/lib/adapters/cultureBoxAdapter.ts / POST /api/sync/culture-box) — the other two are evaluated (structured-data availability, robots.txt permission, current reliability) in their integrationNote but intentionally not yet implemented; Gravity's /events/ page 404s and its sitemap is over a year stale, and Den Anden Side's own site carries no event content at all (it defers entirely to Resident Advisor). Resident Advisor, AllEvents, Billetto, Eventbrite and the Facebook groups are discovery/verification/link references only — automated ingestion from them is deliberately not implemented until a permitted access method (API, confirmed ToS allowance) is confirmed and documented on that source's integrationNote. Do not add scraping for them without doing that first. /about renders this table for anyone non-technical.

Adding a new source

See SOURCE_ONBOARDING.md for the full process (DISCOVER → DIAGNOSE → IMPLEMENT → TEST → VALIDATE → FIX/RETRY → MERGE GATE → PRODUCTION VERIFY) and the explicit conditions that require stopping to ask a human. The permanent tooling it's built on: .github/workflows/inspect-source.yml (parameterized read-only diagnostics — source inventory, discovery-queue/source-links inspection, health, dedup simulation, lock status, reachability, snapshot), .github/workflows/validate-source.yml (parameterized branch validation — lint/tests/build plus an optional live Preview sync/idempotency/ regression check), and scripts/scaffold-source.mjs (adapter/test/sync-workflow skeleton generator). None of these are source-specific or one-off — a new source never needs its own diagnostic or preview-verification workflow file.

Quality gate & dedup

runIngestionPipeline() (adapters/pipeline.ts) is what a real adapter's output would flow through: required-field validation, venue/artist normalization, genre classification (evidence hierarchy, never inferred from title alone as primary evidence), fuzzy duplicate detection against existing events, and a confidence-based publish decision (auto-publish / review queue / hold — never auto-publish below high confidence). The /admin page's "Add event from URL" tool runs a real, best-effort Open Graph extraction through this same pipeline so the review UI reflects actual pipeline output, not a mockup.

Images

Per spec, images are optional and never required for the site to look finished — no seed event, venue or festival currently sets one. EventRecord.imageUrl / FestivalRecord.imageUrl exist in the data model and are already wired into Open Graph tags and JSON-LD when present; rendering an <img> on the event/venue/festival detail pages once real images exist is a small, isolated follow-up (would also need images.remotePatterns in next.config.ts for external hosts).

Contact & Suggest an event

/contact and /suggest-event are simple forms (ContactForm.tsx, SuggestEventForm.tsx) that POST to /api/contact and /api/suggest-event respectively. Each Route Handler validates the submission server-side, rejects obvious bot traffic via a honeypot field and a simple in-memory rate limit (src/lib/rateLimit.ts), and sends the submission as an email through Resend (src/lib/email.ts) — no database write, no automatic publication. A suggestion arrives as an email that an admin reviews by hand, typically by pasting the sender's link into the existing /admin "Add event from URL" tool, which already runs it through the review pipeline into the discovery queue.

This deliberately avoids needing a real mailbox at [email protected]: Resend sends on behalf of a verified sending domain/subdomain, and delivers to whatever CONTACT_RECIPIENT_EMAIL you already have (e.g. a personal address) — see .env.example for the three required server-side-only env vars (RESEND_API_KEY, CONTACT_RECIPIENT_EMAIL, CONTACT_FROM_EMAIL). None of them are ever sent to the browser.

Scheduling

.github/workflows/sync-hangaren.yml calls POST /api/sync/hangaren and .github/workflows/sync-culture-box.yml calls POST /api/sync/culture-box on a schedule — both sources refresh automatically once deployed and configured, no manual trigger required. The two workflows are identical in structure/safety model; only the endpoint, cron offset and concurrency group differ.

• Frequency: every 6 hours for both — Hangaren at cron: "0 */6 * * *", Culture Box at cron: "30 */6 * * *" (offset by 30 minutes purely to spread load across two separate GitHub Actions jobs; not a correctness requirement, since each source has its own advisory lock key — see "Concurrent runs" below). Matches syncFrequency in src/lib/data/sources.ts for src-hangaren and src-culture-box. Both are also runnable on demand via their workflow's workflow_dispatch trigger (e.g. to retry after fixing something).
• Retry behavior: three layers, deliberately not aggressive at any one of them. (1) Each workflow's curl retries the HTTP call itself up to 3 times (15s apart) for transient network issues reaching the deployment. (2) The adapter (hangarenAdapter.ts / cultureBoxAdapter.ts) retries its own fetch once after a 2s delay — one blip shouldn't flag a healthy source as failed. (3) Beyond that, a persistently failing source is left for the next scheduled run 6h later (or a manual workflow_dispatch) rather than retried in a tight loop.
• Failure logging: every sync outcome is written to sources.lastError/lastSuccessfulSync in Postgres (visible on /admin) regardless of how it was triggered. The API route also returns a non-2xx status for failed and zero_events outcomes specifically (never for a normal ok or a benign skipped_concurrent), so the calling workflow run itself goes red in GitHub Actions' own history/notifications — that's the externally-visible failure log, not just a console line inside the process.
• Secrets: SYNC_TRIGGER_TOKEN must be set twice — as a GitHub Actions repository secret (so the workflow can send it) and as an environment variable on the actual deployment (so the route can check it); the workflow also needs a repository variable SYNC_BASE_URL pointing at the deployed app. Both are one-time setup after deploying — see the comment at the top of the workflow file. sync-culture-box.yml reuses these same three repository-level secrets/variable (SYNC_BASE_URL, SYNC_TRIGGER_TOKEN, VERCEL_AUTOMATION_BYPASS_SECRET) rather than needing its own. ADMIN_USERNAME/ADMIN_PASSWORD (below) are separate and only gate /admin.
• Concurrent runs: two layers. Each workflow's own concurrency: group ( / ) serializes overlapping runs of that one workflow. The real guarantee is inside the app: () takes a Postgres advisory lock (, cluster-wide, not just per-process, keyed by source id) for the duration of the sync and skips outright (, HTTP 200 — not an error) if another sync for the same source is already running, however it was triggered (scheduled, manual, or a retried request landing twice). Because the lock is keyed per source, Hangaren and Culture Box runs never contend with each other even if they land at the same time.

Admin access

/admin (the review UI) and every /api/admin/* route (publish/edit/hide events, resolve the discovery queue) are gated behind HTTP Basic Auth in src/proxy.ts, checked against ADMIN_USERNAME/ADMIN_PASSWORD env vars using a timing-safe comparison. If either is unset, access is denied outright (fails closed, never silently open). This is deliberately not a user- management system — a single shared credential pair is the simplest production-appropriate stopgap for a small internal tool, per the brief. /api/sync/[source] is unaffected — it keeps its own separate x-sync-token check so the scheduler can call it without a Basic Auth prompt. Public pages are untouched.

Shows source health, the discovery review queue, and the "Add event from URL" tool described above.

Testing

npm run test runs the vitest suite: chronological sorting, Tonight/weekend filters and their midnight-crossing edge cases, venue alias resolution, artist normalization, duplicate detection tiers, genre evidence confidence, canonical source priority, the quality gate, calendar export (including DST-safe UTC conversion and overnight events), manual-override field protection, the Hangaren HTML parser against a real recorded response, sync-time merge-decision logic (src/lib/sync.test.ts), and the production-bootstrap safety property that no fabricated source sync-health can reach a production row (src/lib/sourceRegistry.test.ts).

Two separate DB-backed proofs (not part of npm test, since both need a live Postgres):

• npm run db:verify-sync live-fetches the real source, then runs the real runSourceSync/database code path through new-event, duplicate, changed-date/time, changed-lineup, provenance-persisted-at-publish, manual-override-survives-sync, source-failure, zero-events-anomaly, and concurrent-runs scenarios, asserting against the actual database each time. Idempotent — cleans up its own prior run before each execution.
• npm run db:verify-bootstrap proves the exact production-prep sequence works against a genuinely empty database: it drops the app's own tables, re-runs migrations from scratch, runs the production bootstrap, asserts zero demo events/discovery-items and that the required registry rows exist with neutral health, runs one real live Hangaren sync, asserts the results are real and attributed correctly, then re-runs the bootstrap a second time and asserts nothing duplicated and the source's now-real sync history wasn't reset. Destructive — refuses to run unless DATABASE_URL looks local or ALLOW_DESTRUCTIVE_RESET=1 is set; never point it at a database you care about.