Sepsis Early Warning

A model that scores ICU patients every hour from routine vitals and labs and tries to flag sepsis up to 12 hours before onset.

Live demo | Write-up | Portfolio

It's trained on the PhysioNet 2019 Challenge data and scored with the Challenge's own utility metric.

I wanted to build something in healthcare and learn the tools along the way. The task, data and scoring all come from the 2019 PhysioNet/CinC Challenge paper (Reyna et al., cited at the bottom). It was a heavy task for a laptop. Mine kept breaking.

Live demo: replays held-out patients hour by hour and shows when the alert would have fired.

Results

4,441 held-out patients, split by patient so none of them were seen in training. The alert threshold was picked by cross-validation on the training patients only.

Metric	Value
Normalized utility (official Challenge metric)	0.424
AUROC / AUPRC (hourly)	0.817 / 0.092
Septic patients flagged in the useful window (−12h to +3h of onset)	370 / 398 (93%)
Median warning before onset, among those flagged	12 h
Non-septic patients with at least one alarm	2,381 / 4,043 (59%)
Septic patients who also alarmed before the useful window	271 / 398 (68%)

Read the last two rows before the first four. The model catches most septic patients partly because it alarms on a lot of people. The 12h median is also the most the window allows: many of those patients were already alarming when the window opened.

The threshold cross-validation picked (0.32) sits at the peak of the test curve.

How it works

flowchart LR
    A[".psv files<br/>one row per ICU hour"] --> B["load.py"]
    B --> C["features.py<br/>causal features"]
    C --> D["train.py<br/>patient split,<br/>CV-tuned threshold"]
    D --> M[("model.joblib")]
    D --> E["evaluate.py<br/>utility, AUROC, lead time"]
    M --> F["app.py<br/>FastAPI POST /score"]
    F --> G["build_pages.py<br/>pre-scores demo patients"]
    G --> H["GitHub Pages"]

Python 3.13, pandas, scikit-learn's HistGradientBoostingClassifier, FastAPI, a plain JS demo page, pytest, ruff, Docker, GitHub Actions.

Choices that matter:

• Scoring uses the Challenge's own compute_prediction_utility, copied unchanged into src/sepsis_utility.py. It gives the most credit for alerts 6 to 12h before onset, penalizes late or missed cases hard, and charges 0.05 for every false-alarm hour.
• Features at hour t only use data up to t (forward-fills, rolling and expanding windows). tests/test_features.py appends and corrupts future hours and checks that nothing earlier changes.
• Labs are sparse, and when one gets ordered says something. Each channel has a "measured this hour" flag and an "hours since last measured" count.
• Train/test and the CV folds are both split by patient, so one stay never ends up on both sides.
• The threshold is tuned on 3-fold out-of-fold predictions. Tuning on the final model's own training scores would be tuning on overconfident numbers: in-sample utility is 0.67 against 0.42 on test.
• Lead time only counts alerts inside the window the metric rewards. An alarm 30 hours before onset usually fired at admission, before anything predictive happened, so those are reported separately.

Limitations

• The alarm burden is high. At the utility-optimal threshold, most non-septic patients alarm at least once. The metric only charges 0.05 per false-positive hour so it tolerates this. A real deployment would need alert suppression or a stricter threshold.
• Don't compare 0.424 with the Challenge leaderboard (best entries scored about 0.36). Those were scored on hidden test sets that included a hospital no team had data from. This is an internal split of one hospital system, which is much easier.
• Trained on 17,762 of the 20,336 patients in training set A, the most my laptop could run without breaking down. Training set B (the second hospital) wasn't used. Training on A and testing on B is the external-validation experiment this still needs.
• It's a gradient-boosting baseline. Top entries added sequence models and per-hospital calibration, and the probabilities here aren't calibrated.
• Research demo only. Not a medical device and not validated for clinical use.

Setup

make setup    # pinned dependencies
make run      # demo + API at http://localhost:8000
make test
make lint

Without make (plain Windows, for example):

pip install -r requirements-dev.txt
export PYTHONPATH=src            # PowerShell: $env:PYTHONPATH="src"
uvicorn app:app --app-dir src --reload
pytest -q

The trained model (artifacts/model.joblib) is committed, so the demo runs without training anything.

This is the beginning of the README. Read the rest at the original. Read the full README ↗