Berlin Rent Predictor

Quick start · Results · Limitations · Model card

Enter a flat, get a rent estimate. Trained on about 10,000 real ImmoScout24 Berlin listings and served through a Streamlit UI and a FastAPI endpoint, with tests, Docker and CI.

Berlin listings cluster by location, so a random train/test split leaks neighbourhood information and flatters the score. This project also evaluates with a postal-code grouped split, where every test postal code is unseen during training.

The elevation above is not decoration: every block is a real flat from data/processed/berlin_processed.csv, its height set by that flat's monthly rent and its width by its floor area, sampled evenly across the market. Redraw it with python .github/assets/make_assets.py.

---

Pipeline

---

Quick start

pip install -e ".[dev,train,ui]"
python scripts/train.py            # trains, compares models, saves models/
python scripts/tune.py             # tuned log-rent model + 90% intervals (overwrites models/)
python scripts/analyze_errors.py   # error by district, size and price range
python scripts/explain_model.py    # SHAP feature importance
pytest                             # run tests
streamlit run app/streamlit_app.py # web UI at http://localhost:8501

Or serve the REST API instead of the UI:

uvicorn berlinrentml.api.main:app --port 8000

curl -X POST localhost:8000/predict -H "Content-Type: application/json" \
  -d '{"livingSpace": 60, "rooms": 2, "geo_plz": "10115", "geo_bln": "Mitte"}'

{"predicted_rent": 1197.21, "interval_low": 837.13, "interval_high": 1711.99, "model_name": "LGBMRegressor", "timestamp": "..."}

Same flat in Marzahn ("geo_plz": "12619", "geo_bln": "Marzahn") returns about 613 €. Interactive docs are at /docs.

Results

10,388 Berlin listings after cleaning, 80/20 split. LightGBM wins on both splits.

• LightGBM: R² 0.880 on the random split, 0.836 on unseen postal codes. About 62% lower error than the median baseline.
• Leakage is real: linear regression goes from €186 to €291 once test postal codes are unseen. The final model is picked by grouped-split MAE, not random-split MAE.
• Where it fails: the model is least accurate in expensive central districts and for large flats, which it under-prices.

Beyond a baseline

python scripts/tune.py goes further than a single fit. It splits by postal code into train, calibration and test sets, so every number is for neighbourhoods the model never saw.

LightGBM variant	MAE (€)	R²
Default, raw rent target	195.6	0.835
Log-rent target	193.3	0.832
Log-rent, tuned (40-candidate search, postal-code grouped CV)	190.7	0.831

The gains are small, and the README says so on purpose: the limit is the data (asking rents, no address, no photos), not the algorithm.

• Prediction intervals. Split-conformal intervals from out-of-fold residuals on unseen postal codes. The 90% interval covered 88.5% of test listings, with a median width of about €615. Coverage is weaker in rare, expensive districts (Grunewald: 59%), which the notebook shows.
• Experiment tracking. Each tuning run is logged to MLflow (mlflow ui --backend-store-uri sqlite:///mlflow.db).
• Drift monitoring. The API logs requests to logs/requests.jsonl. python scripts/check_drift.py computes the Population Stability Index of live inputs against the training data.
• Notebooks. 01 · EDA and the leakage trap and 02 · Modeling and uncertainty, executed with outputs.
• Reproducibility. make all runs train, tune, analysis and tests. CI also trains on synthetic data, starts the API and calls /predict as an end-to-end smoke test.

What is inside

Area	What it does
Data	Loads the Kaggle CSV, filters to Berlin, maps columns, validates schema and ranges
Cleaning	Removes impossible values, caps rent outliers, handles missing data
Features	Building age, size and age bins, log size, rooms per m², amenity score
Evaluation	Random split and GroupedSplit by postal code, MAE, RMSE, R², share within €50/100/200
Models	Mean, median, linear, ridge, random forest, LightGBM, XGBoost, lasso, elastic net
Serving	FastAPI with validated requests, /health, /model/info, /predict
Quality	pytest suite, GitHub Actions CI, Dockerfile, model card

Install

Requires Python 3.10 or newer.

git clone https://github.com/arjun05-tf/berlin-rent-predictor.git
cd berlin-rent-predictor
pip install -e ".[dev,train,ui]"

Get the data

Download immo_data.csv from Kaggle: Apartment rental offers in Germany and place it in data/raw/. Or use the CLI:

kaggle datasets download -d corrieaar/apartment-rental-offers-in-germany -p data/raw --unzip

The dataset has 268,850 listings across Germany from 2018 to 2020. The pipeline keeps the 10,406 Berlin rows. The CSV is gitignored. See data/DATASET.md.

Common commands

python scripts/train.py            # full pipeline, saves models/final_model_*.joblib
python scripts/analyze_errors.py   # error by district, size and price range
python scripts/explain_model.py    # SHAP feature importance
pytest                             # run tests

Deploy to Vercel

The repo is Vercel-ready: public/index.html is the web form, api/index.py serves the FastAPI app as a serverless function (/api/index?r=predict, /api/index?r=health), and the 1.5 MB model files in models/ ship with it. Runtime dependencies are pinned in requirements.txt to the versions the model was trained with.

npm i -g vercel
vercel          # preview deploy, then: vercel --prod

Or import the GitHub repo at vercel.com/new. No build settings are needed. If you retrain, run python scripts/export_options.py to refresh the dropdown values, then commit models/ and public/options.json. The Streamlit app (app/) is for local use, because Vercel cannot host long-lived Streamlit servers.

Docker

Train first, because the image copies models/.

docker build -t berlin-rent .
docker run -p 8000:8000 berlin-rent

Project layout

src/berlinrentml/
  data/        loading, validation, cleaning
  features/    feature engineering, preprocessing
  modeling/    training, evaluation, split strategies
  api/         FastAPI service
scripts/       train, error analysis, SHAP
tests/         unit and API tests

Limitations

• The data is from 2018 to 2020. Predictions reflect past rents, not today's market.
• Offered rents are asking prices, not signed contracts.
• Postal codes unseen in training fall back on the other features, and accuracy drops (about €31 MAE worse than the random split).
• Intervals under-cover in rare, expensive districts, and the model under-prices large high-end flats.

See MODEL_CARD.md for intended use and ethical considerations.

Troubleshooting

Problem	Fix
Model not loaded from /predict	Run python scripts/train.py first
No data file found	Put immo_data.csv in data/raw/
UnicodeEncodeError on Windows	Set PYTHONUTF8=1 before running scripts
Docker build fails at COPY models/	Train first so models/ has the .joblib files

License

MIT. Data from ImmoScout24 via Kaggle, see the dataset page for its terms.