Run365Days
In 2021 I ran every single day. This repository is the analytics side of
that challenge: a Python package that parses 365 Garmin exports, a daily
weight log and a year of Hong Kong Observatory weather, and a dashboard that
puts the results on one page.
Live dashboard: https://python-project-run365-days.vercel.app/ (React
on a Flask + GraphQL API) · https://dcwhung.github.io/Python-Project-Run365Days/
(the same React app in static mode)
It started as a handful of scripts in 2022 and has been rebuilt three times
since: into a tested package, into a feature-organised codebase with a
static dashboard and CI, and now into a React dashboard served in two ways
from one codebase, with a Python GraphQL API behind the Vercel deployment.
Contents
What it does
| Input | What the package extracts |
|---|
| Garmin TCX / GPX / KML exports (365 runs) | distance, time, pace, calories, cadence, altitude, per-point speed and ambient temperature, GPS track |
| Daily weight log (365 entries) | weight in lb and kg, BMI, day-over-day change, monthly and weekday trends |
| HKO daily extract, hourly history and warning database | temperature, humidity, wind and rain at run time; warning signals in force that day |
| Garmin account export (trimmed) | activity summaries, daily wellness and sleep for the challenge period, reserved for the next version |
The dashboard combines them into an overview (KPIs, distance heatmap,
monthly totals, personal bests, training load), a year-in-review
infographic, an animated per-run view with route playback and synced
charts, and dedicated performance, weight, weather-impact and training-load
pages.
Stack
| Area | Choice |
|---|
| Language | Python 3.10+ (X | None unions, dataclasses) |
| Parsing | xml.etree.ElementTree for TCX / GPX / KML, BeautifulSoup + lxml for HTML embedded in KML and for scraping |
| Numerics | pandas and numpy for lap tables, descriptive statistics and date ranges |
| Storage | SQLite via SQLAlchemy 2.0 (typed ORM models), generated at build time |
| API | Flask 3 + Strawberry GraphQL (code-first schema), read-only over the SQLite export |
| Front end (v3, in progress) | React 19 + TypeScript on Vite, TanStack Query + graphql-request with GraphQL Codegen, Tailwind CSS 4; runs in api or static data mode |
| HTTP | requests |
| Front end | Single HTML page, vanilla JS, Chart.js 4 from cdnjs, Canvas for the route map |
| Packaging | setuptools with a src-layout, console scripts in pyproject.toml; core dependencies are the API runtime only, the parsing stack sits in the pipeline extra |
| Quality | ruff (lint, import order, format, docstrings), pytest, pre-commit |
| Delivery | GitHub Actions, GitHub Pages |
Repository layout
pyproject.toml package metadata, console scripts, ruff and pytest config
src/ Python package, imported as `run365days`, one sub-package per feature
activities/ Activity / TrackPoint models, parsers/ (tcx, gpx, kml), metrics (MET)
weather/ weather models, collectors/ (hko_daily, hourly, sun_moon, warnings)
weight/ daily weight parsing, year table, summaries
dashboard/ builder (shared run calculations), stats (year aggregates)
common/ config (all paths and constants), geo, time
cli/ process_activities, collect_weather, export_data, export_schema
tests/ pytest suite, one file per feature module
data/
raw/garmin/{tcx,gpx,kml}/ 365 Garmin activity exports for 2021
raw/garmin/*.json activity summaries, daily wellness, sleep
raw/weather/ HKO daily extract, hourly history, warnings, sun and moon
raw/weight/ daily weight log, original tracking spreadsheet
processed/ run365.db, static/*.json, JSON Lines (git-ignored)
frontend/ React dashboard (v3): src/data (api + static sources), src/views, src/app
docs/ architecture, data pipeline, changelog, roadmap
legacy/ original 2022 scripts, reference only
.github/workflows/pages.yml CI and deployment
LICENSE MIT, with a personal-data exclusion for data/
Getting started
git clone https://github.com/dcwhung/Python-Project-Run365Days.git
cd Python-Project-Run365Days
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
pre-commit install # optional: ruff on every commit
pytest tests # backend tests
run365-export # data/ -> data/processed/{run365.db, static/}
flask --app run365days.api.app:create_app run # GraphQL at http://127.0.0.1:5000/api/graphql
cd frontend && npm install && npm run dev # dashboard at http://localhost:5173
The package finds its data relative to the repository. To point it
elsewhere, set RUN365_DATA_DIR to a directory with the same
raw/ and processed/ structure.
Dependency extras
Installing the package bare gives you the API runtime only — Flask,
SQLAlchemy and Strawberry. That is deliberate: requirements.txt installs
the package, so the core list is exactly what Vercel bundles into the
api/graphql.py function, and pandas, numpy and lxml would add roughly
165 MB to every deployment for code the API never imports.
| Install | Gets you | Use it for |
|---|
pip install -e . | Flask, SQLAlchemy, Strawberry | serving the GraphQL API |
pip install -e ".[pipeline]" | the above plus pandas, numpy, lxml, BeautifulSoup, requests, pytz, python-dateutil | run365-activities, run365-weather, run365-export |
pip install -e ".[dev]" | [pipeline] plus pytest, ruff, pre-commit | working on the repo |
Running any console script other than run365-schema without the
pipeline extra raises ModuleNotFoundError. tests/test_packaging.py and
tests/test_api_imports.py keep the two sides of this split honest.
Command-line tools
Installing the package registers the console scripts listed below (the
full set is declared under [project.scripts] in pyproject.toml). Each
can also be run as python -m run365days.cli.<module>.
| Command | What it does | Reads | Writes |
|---|
run365-activities --format all --year 2021 | Parse every activity file into normalised records | data/raw/garmin/{tcx,gpx,kml}/ | data/processed/activities_<fmt>.jsonl |
run365-weather --source all --year 2021 | Scrape hourly weather, HKO warnings, the HKO daily extract and sun / moon times | the web | data/raw/weather/*.json |
run365-export [--points 600] [--skip-db] [--skip-static] [--static-dir DIR] | Parse everything once and write the processed data set for the API and the static build | data/raw/** | data/processed/run365.db and data/processed/static/*.json |
run365-schema [--check FILE] | Print the GraphQL SDL, or verify a file matches it | the Strawberry schema | stdout |
GraphQL API
run365-export # build data/processed/run365.db once
flask --app run365days.api.app:create_app run # GraphiQL at http://127.0.0.1:5000/api/graphql
The schema exposes meta, activities(fromDate, toDate, minKm, hasGps),
activity(id) with a downsampled track(points), weight, weather,
warnings and a year aggregate (totals, monthly, weekly, daily distance,
training load, personal bests). api/graphql.py exports the same app for
Vercel; RUN365_DB_PATH points it at the bundled database.
Dashboard
Live: https://python-project-run365-days.vercel.app/ (API mode on
Vercel) and https://dcwhung.github.io/Python-Project-Run365Days/ (static
mode on GitHub Pages). See docs/deployment.md.
frontend/ is a Vite + React 19 + TypeScript app styled with Tailwind CSS 4.
Charts use Chart.js; the route map and the per-run charts are drawn on
Canvas. Views:
- Overview: KPIs, distance heatmap (click a day to open it), monthly
distance and pace, recent runs with weather and HKO warning icons,
personal bests, 42-day / 7-day training load, weight against distance,
temperature against pace, weather strip.
- Year in Review: the year-end infographic with days, distance, monthly
hours, calories and weight change.
- Activity: any of the 365 runs with a pace-coloured route that draws
itself from start to finish (play, speed, scrub, hover) and synced
elevation, pace, cadence and temperature charts with a live readout.
Indoor runs show the charts without a map.
- Performance, Weight, Weather Impact, Training Load: one analytics page
per topic, each with KPIs, charts and a table.
- Activities: sortable, filterable table with CSV export.
- Settings: landing view, playback speed, weight unit, height for BMI,
pace colour bounds; stored in the browser.
Data modes
The app talks to one DataSource interface (frontend/src/data/types.ts).
VITE_DATA_MODE | Source | Aggregates |
|---|
api (default) | GraphQL endpoint; documents are type-checked against frontend/schema.graphql (written by run365-schema, checked in CI) | computed by the API |
static | JSON written by run365-export --static-dir frontend/public/data | computed in the browser by a TypeScript port of dashboard/stats.py, pinned to the same test values |
Views never know which mode is active.
cd frontend
npm run dev # api mode, /api proxied to Flask on :5000
npm test # vitest (runs codegen first)
npm run build # api mode -> dist/
npm run build:static # static mode -> dist/, reads /data/*.json
How the data flows
raw Garmin files ──► activities.parsers ──► Activity[] ─┐ ┌─► run365.db ──► Flask + Strawberry ──► React (api mode)
weight log ──► weight.analysis ──► WeightRecord[] ─┼─► export.records ─┤
weather JSONL ──► (loaded as rows) ─┘ └─► static/*.json ──────────────────► React (static mode)
TCX is the primary source for every run. GPX is joined by activity id for
its per-point ambient temperature. For each run the export attaches the
hourly observation nearest to the start time and the HKO warning signals in
force that day, using the pure helpers in dashboard/builder.py. One
intermediate structure, ExportRecords, is written both as SQLite (for the
API) and as split JSON (for the static build), so the two modes always agree. The full account, including the
quirks of each source, is in docs/data-pipeline.md
and the reasoning behind the layout in
docs/architecture.md.
Testing and code quality
pytest tests
ruff check src tests
ruff format --check src tests
tests/ is the inventory of what the Python suite covers. This page
points at that directory instead of restating it, so there is nothing
here to fall out of sync. Among the areas exercised there: the geo and
time helpers, MET and calorie maths, weight parsing (including the
undated-last-line quirk), every dashboard builder function, the year
statistics, the export records, SQLite and JSON writers, the CLI
serialiser, the TCX, GPX and KML parsers, the HKO daily, hourly and
warning collectors together with the raw-row contract they share with
the export layer, and the GraphQL API driven through a Flask test
client against a temporary database.- Front end: the Vitest modules sit beside the code they cover under
frontend/src/ and are the inventory in the same way. Among them: the
TypeScript stats port (pinned to the Python numbers), the static JSON
mappers and source, the API source, data-mode resolution, formatting
and weather helpers, track downsampling, GraphQL error mapping, every
view's model module, the activity series builder and the activity view
itself, and all nine views rendered against a fake data source.
- ruff enforces pycodestyle, pyflakes, isort, pyupgrade, bugbear,
simplify, pep8-naming and Google-style docstrings on every public
symbol. Line length is 100.
.pre-commit-config.yaml runs the same checks locally; legacy/ is
excluded everywhere.
Screenshots
| Activity: route playback with synced charts | Year in Review |
|---|
| |
Continuous integration and deployment
.github/workflows/pages.yml gates every push and pull request it runs on
with the Python checks (lint, format, tests and GraphQL schema sync) and the
frontend checks (lint, typecheck, unit tests and a build in both data modes).
That file is the source of truth for which branches it runs on and for the
steps of each job; they are not repeated here, so there is no second copy to
keep in sync.
The static dashboard is built and deployed to GitHub Pages from one branch
only -- develop -- and a gated branch that is not the deploy source stops
after the checks. docs/deployment.md covers that split,
the github-pages environment rule it depends on, and every step needed to
move the deploy source.
Vercel (API mode)
vercel.json describes the second deployment: scripts/vercel-build.sh
installs the package with uv, runs run365-export --skip-static to produce
data/processed/run365.db, and builds the React app in API mode.
api/graphql.py is a Python serverless function that imports the Flask app;
the database is bundled into it with includeFiles. Rewrites send
/api/* to the function and everything else to the SPA. The production
branch should be develop, but Vercel keeps that setting in the project
dashboard rather than in the repository, so it cannot be read or changed
from this checkout -- confirm it there after any change to the branch
layout. docs/deployment.md has the full
configuration and the list of things the first deployment taught.
Generated files (data/processed/, frontend/public/data/, frontend/dist/)
are git-ignored and rebuilt on every deploy.
Versioning and branches
| Branch | Tag | Contents |
|---|
| (tag only) | v1.0.0 | Original 2022 scripts and raw data |
release/v1.5 | v1.5.0 | First modular package with tests |
release/v2 | v2.0.0 | Static dashboard mockup on top of v1.5 |
develop | v3.0.0 | Feature-organised package, SQLite + JSON export, Flask + Strawberry GraphQL API, React dashboard in two data modes, Vercel + GitHub Pages deployments |
Release branches are frozen snapshots. New work lands on develop through
pull requests, which the CI workflow gates, and develop is what deploys.
master is the repository's default branch and is gated by the same CI
checks, but it does not deploy; moving the deploy source to it is a later
decision, and docs/deployment.md lists every step
that switch needs.
The history is in docs/CHANGELOG.md. Tags and GitHub
Releases are created by the manual "Tag release" workflow, which takes its
notes from the changelog.
Privacy
data/ contains my own activity, weight and scraped weather records and is
kept in the repository so the project builds end to end. The Garmin account
export was trimmed for v3.0.0 to the three files the roadmap needs;
earlier tags still contain the full export (profile, device and consent
records) because the history has not been rewritten. Treat everything under
data/ as personal data: see the exclusion at the end of LICENSE.
Documentation
Licence
Code is released under the MIT License. The personal records
under data/ are excluded; see the note at the end of that file.
