Fomo NYC
fomo.nyc is a free, community-built interactive map of upcoming events in the NYC area.
About
It works by visiting the websites of parks, museums, music venues, etc., identifying any upcoming events, and displaying them on a map. My hope is that this becomes a useful resource for people to find events they are interested in and to engage with their local communities.
The codebase is city-agnostic — NYC is just the default deployment. Every city/region/branding specific (map view, timezone, geography, prompt wording, site name, per-platform crawl behavior) lives in config/<FOMO_CITY>.yaml (default config/nyc.yaml) and gitignored pipeline/sources/ plugins, so you can fork it for your own city by editing config rather than engine code.
Project Structure
Main Directories
src/js/ JavaScript modules (IIFE pattern, loaded in order)src/css/ CSS modules (imported via index.css)src/data/ Event and location JSON data (generated by pipeline)src/api/ PHP API endpointssrc/admin/ PHP admin dashboardsrc/images/ SVG assetssrc/fonts/ Self-hosted MapLibre glyph fonts
- Generated by
npm run build (prod) or npm run dev (dev)
- Contains everything needed for the server's
public_html/ directory
- Uploaded to production via
python scripts/upload_public_html.py
main.py Main entry point - orchestrates the complete workflowcrawler.py Crawls websites using Crawl4AI, stores content in databaseextractor.py Uses Gemini AI to extract structured event data
- Requires
GEMINI_API_KEY environment variable (set in .env file)
processor.py Parses extracted data, enriches with location coordinatesmerger.py Deduplicates events into final events tableexporter.py Generates JSON files for the website (src/data/)uploader.py Uploads data files to FTP serverdb.py Database operations (CRUD for crawl runs, results, events)
schema.sql Complete database schemasetup.py Creates empty database tablesmigrate_schema.py Applies schema changes to existing database/database/backups/ Database backup files
Build System
The frontend is built with esbuild:
npm run dev # Dev build + watch (unminified, symlinks, stable filenames)
npm run build # Prod build (minified, content-hashed, full asset copies)
- JS modules are concatenated in load order and minified (not bundled — they use the IIFE pattern with globals)
- CSS
@import chain is resolved and minified
- Flatpickr is installed via npm and bundled into the output
build.js reads config/<FOMO_CITY>.yaml and injects the city config: a window.__CITY__ global (map/timezone) prepended to the JS bundle, plus {{TOKEN}} branding replacement in index.htmldist/ contains the complete deployable output: HTML, JS/CSS bundles, plus copies of data/, images/, fonts/, api/, admin/
Data Pipeline Flow
All data flows through the database (crawl_runs → crawl_results → crawl_events → events):
- Crawl → Query
websites table for due sites, crawl and store in crawl_results.crawled_content
- Extract → Use Gemini AI to extract structured tables, store in
crawl_results.extracted_content
- Process → Parse tables, enrich with location data from
locations, store in crawl_events
- Detail crawl → Visit individual event URLs to fill in missing descriptions/tags/emoji on
crawl_events
- Merge → Deduplicate
crawl_events into final events table
- Export → Generate
src/data/*.json from events table
- Upload → Push JSON files to FTP server
The day-to-day entry point is the /run-pipeline workflow (.claude/commands/run-pipeline.md), which wraps pipeline/main.py with review and fix steps.
Deployment
npm run build # Build to dist/
python scripts/upload_public_html.py # Upload dist/ to server
How You Can Help
Database
The project uses a MariaDB/MySQL database to store locations, websites, and crawl data.
Initial Setup
New developers should restore from a database backup rather than starting with an empty database:
# 1. Create the database
mysql -u root -e "CREATE DATABASE fomo CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci"
# 2. Restore from backup
mysql -u root fomo < database/backups/fomo_backup_YYYYMMDD.sql
Creating Backups
# Windows (XAMPP)
"C:/xampp/mysql/bin/mysqldump.exe" -u root fomo > database/backups/fomo_backup_YYYYMMDD.sql
# Linux/Mac
mysqldump -u root fomo > database/backups/fomo_backup_YYYYMMDD.sql
Schema Updates
If the schema has changed since your backup, run migrations:
python database/migrate_schema.py
Acknowledgements
