CI Observatory

See how your CI/CD really works. One command reads the GitHub Actions history of your repositories, public or private, and writes a single HTML report: what is slow, what fails and why, which tests run twice, how fast you ship, how much AI is involved, and what to fix first.

Try it on any repository · Live demo · Website · Documentation

curl -LsSf https://kirbac1.github.io/ci-observatory/install.sh | sh
ci-observatory --org your-org --open

The demo is built from public data of 19 repositories in the aiven and Aiven-Open GitHub organisations, from 3 September to 18 October 2026. It is not affiliated with or endorsed by Aiven.

What you get

One self-contained index.html with seven sections, picked one at a time from a sidebar that shows each section's headline number. Every chart answers one question in plain words, and picking a repository (in the sidebar, or by clicking it in any chart) focuses every section on it.

Section	The question it answers
Overview	Which repositories are fast and reliable? When do runs start? How many pass each day?
Delivery (DORA)	How often do you ship, how long from pull request to release, how often does a release need a fix, and how fast is a broken main branch fixed? Each rated Low to Elite.
Health check	Why do jobs fail? Where does job time go? Are the safety basics (read-only tokens, pinned actions, time limits) in place?
Inside a job	For any job: where did the time go, which steps ran, what did the log say, and if it failed, why?
Test audit	On one recent pull request per repository, which tests ran more than once, and was the repeat useful or waste?
AI impact	Which AI tools and models write and review code, how much CI time involves AI, and do AI-written changes run into more problems?
Solution box	Every problem found, with the gain and how to fix it (often ready YAML), in the order to tackle them. Fixes you have already tried and timed can be added with --fixes, and show their before and after.

What each chart means, and what to do about it, is in Reading the report.

Install

You need the GitHub CLI, logged in (gh auth login), or a token in GH_TOKEN. Everything else comes with the installer.

macOS and Linux, one line:

curl -LsSf https://kirbac1.github.io/ci-observatory/install.sh | sh

It installs uv if you don't have it (uv brings its own Python, so your system Python is never touched), then installs the ci-observatory command with uv tool install. No sudo. The script is short and readable; read it first if you like.

Run once without installing anything (anywhere uv runs):

uvx --from git+https://github.com/kirbac1/ci-observatory ci-observatory --org your-org --open

With pipx, if that's what you use:

pipx install git+https://github.com/kirbac1/ci-observatory

Pinning a version, upgrading, uninstalling and installing behind a proxy are in docs/install.md.

Quick start

gh auth login                          # once
ci-observatory --org your-org --open   # the 12 busiest repositories of an organisation

It prints its progress, takes about 5 to 10 minutes for a dozen repositories, and opens ./ci-observatory/index.html in your browser. A second run reuses the cached job logs and is much faster.

Inside a clone of any GitHub repository, no arguments are needed:

cd ~/code/my-service
ci-observatory --open

Everyday commands

You want	Command
The repository you are in	ci-observatory --open
One or a few repositories	ci-observatory --repo my-org/api --repo my-org/web
The 12 busiest repositories of an organisation	ci-observatory --org my-org
The 25 busiest, private ones only	ci-observatory --org my-org --top 25 --visibility private
Several organisations in one report	ci-observatory --org my-org --org my-other-org
Every repository that ran CI, forks and scheduled-only ones too	ci-observatory --org my-org --all-active --top 100 --min-runs 1
Your own account, private repositories included	ci-observatory --org my-login
A fast first look	ci-observatory --org my-org --quick
The last 60 days	ci-observatory --org my-org --days 60
Skip job logs, the test audit, or AI and DORA	--no-logs, --no-tests, --no-ai
Another output folder or title	--out reports/2026-10 --title "Payments CI"

ci-observatory [--org ORG]... [--repo OWNER/NAME]... [options]

  --org ORG             organisation or user to scan (repeatable)
  --repo OWNER/NAME     repository to scan (repeatable)
  --top TOP             with --org: how many of the busiest repositories (default 12)
  --min-runs MIN_RUNS   with --org: skip repositories with fewer runs (default 10)
  --visibility {all,public,private}
                        with --org: which repositories (default all)
  --all-active          with --org: also forks and repositories nobody pushed to,
                        if they ran CI in the window
  --days DAYS           look back this many days (default 30)
  --out OUT             output folder (default ./ci-observatory)
  --title TITLE         report heading (default: CI Observatory)
  --fixes FILE          a fixes.json of fixes tried and measured on these repositories,
                        shown in the Solution box
  --quick               smaller samples: fewer API calls and log downloads
  --no-logs             skip job logs and the test audit
  --no-tests            skip the test redundancy audit
  --no-ai               skip AI and DORA signals
  --max-calls MAX_CALLS API call budget per step (default 4000)
  --open                open the report in your browser when it is ready
  --version             show the version and exit

Exit codes: 0 done; 1 done, but a limit was reached or some data was missing (the report says what); 2 bad usage, or gh is missing or not logged in.

The web app

The same analysis also runs as a small web service, live at https://ci-observatory-718558656829.europe-north1.run.app/: paste a repository, or an owner such as aiven for all of its public repositories, watch it run, read the report. Public repositories work without signing in; signing in with GitHub (a read-only GitHub App) adds private repositories. The token stays on the server and is revoked at sign-out, and private reports are shown only to the person who asked.

CIO_SERVER_TOKEN=$(gh auth token) uv run --extra server python -m ci_observatory.server   # http://localhost:8000

Configuration, the GitHub App, the security model and deploying to Cloud Run or any Docker host are in docs/web-app.md.

Private repositories

The command does not change. What matters is that your gh login, or the token you hand to gh, can read the repositories. Three ways:

1. Your own gh login (simplest). Add the scopes once, then run as usual:

   gh auth refresh -h github.com -s repo,read:org
   ci-observatory --org my-org --visibility private --open
   

   If your organisation uses SAML single sign-on, also authorise the GitHub CLI for it at https://github.com/settings/applications. Without that, private repositories look like "Not Found".

2. A fine-grained token (least privilege). Give it read-only Actions, Contents and Pull requests on the repositories you choose, then:

   GH_TOKEN=github_pat_... ci-observatory --org my-org
   

3. Inside GitHub Actions. The workflow's own token can read its repository; a GitHub App token can read a whole organisation. See docs/github-actions.md.

Before collecting anything, the tool checks that it can read every --repo and tells you which are private. The step-by-step guide, with SSO, token approval, GitHub Enterprise Server and troubleshooting, is docs/private-repositories.md.

Safe by design

• Read-only. Every request is a GET. Nothing is written to GitHub.
• Your token stays put. With the GitHub CLI installed (the usual case), the tool only runs gh and never sees a token. Without gh, it reads GH_TOKEN and sends it to the GitHub API only, never to log storage or anywhere else.
• Everything stays on your machine. One HTML file and its data, in a folder you choose. No server, no telemetry. Raw logs are cached in ~/.cache/ci-observatory/ and can be deleted at any time.
• People are not named. Authors and reviewers are counted as "people"; only bots and AI accounts appear by name. Log excerpts are cleaned of names, emails, tokens and IP addresses.
• No dependencies. The command needs Python's standard library, plus gh or a GH_TOKEN. The report uses plain SVG and no JavaScript libraries. Only the optional web app adds FastAPI and uvicorn.

The report still describes your repositories (their names, workflow names, cleaned log lines, and the titles of AI-assisted pull requests), so share it the way you would share the repositories themselves.

What it reads, and what it costs

Step	Reads	Typical API calls
1. actions	workflows, runs (newest 1,000 per repository), sampled jobs and steps, workflow files, pull requests, releases	30–60 per repository
2. logs	the latest failed and successful jobs per repository, then every job of one recent pull request commit	15–45 per repository
3. signals	agent instruction files, AI workflows and their runs, pull requests with commits and reviews, default-branch commits, releases	20–80 per repository
4. report	nothing; it builds index.html from the data	0

A dozen repositories take 1,000 to 2,000 calls, well within GitHub's 5,000 an hour. --quick roughly halves that. Each step stops cleanly at --max-calls or on a rate limit, and the report says what is missing.

Documentation

Guide	What's in it
Install	Every way to install, run once, pin a version, upgrade and uninstall
Private repositories	gh scopes, SAML SSO, fine-grained tokens, GitHub Enterprise Server, troubleshooting
GitHub Actions	A weekly report for one repository, or for a whole organisation with a GitHub App
Reading the report	Every section and chart, what it means, and what to do about it
How it works	The pipeline, the data files, how every number is made, DORA proxies, AI detection, limits
The web app	Running the analysis as a service with GitHub sign-in: security model, GitHub App, Cloud Run

Developing

git clone https://github.com/kirbac1/ci-observatory && cd ci-observatory
uv run pytest                                  # unit tests, including the web app against fakes
uv run ruff check && uv run ruff format --check
uv run mypy                                    # strict
sh scripts/build-site.sh && python3 -m http.server -d _site   # the website and demo, at http://localhost:8000

The code is in src/ci_observatory/: one module per step (actions.py, logs.py, signals.py, report.py), a small GitHub client (gh.py, through gh or directly over HTTPS), the command (cli.py) and the web app (server/). The page is in src/ci_observatory/web/: page.html, observatory.css and the obs-*.js modules. To work on the page without calling GitHub, rebuild the demo from the saved data:

uv run python -m ci_observatory.report --data-dir demo/data --out /tmp/report.html && open /tmp/report.html

Every push runs tests on Python 3.11 and 3.14, ruff, mypy, a syntax check of the page scripts, a build of the site and a build of the web app's container. Pushes to main publish the website and the demo to GitHub Pages.

License

MIT