Featurizer

Automated feature engineering for temporal data using PostgreSQL.

Documentation — ccd-ia.github.io/featurizer: Walkthrough · How a feature is computed · Tutorial notebooks · Primitives · Configuration · ADRs · Validation reports

Featurizer implements Deep Feature Synthesis (DFS; Kanter & Veeramachaneni, IEEE DSAA 2015) for relational databases with first-class support for temporal semantics. Given a schema of entities and relationships, it automatically synthesizes hundreds of meaningful features by traversing the entity graph, applying aggregations across relationships, and generating time-windowed statistics.

Why Featurizer?

Feature engineering is often the most time-consuming part of building machine learning models on relational data. Featurizer automates this process by:

• Traversing relationships automatically: Define your entity graph once; Featurizer handles the joins and aggregations.
• Respecting temporal semantics: Point-in-time correct features via as-of joins prevent data leakage in time-series ML.
• Generating pure SQL: No data movement required—features are computed where your data lives.
• Providing extensible primitives: 68 aggregations and 83 transformations out of the box, with a simple API for custom ones.

Quick Example

from featurizer import Featurizer

# Load configuration defining entities and relationships
f = Featurizer("config.yaml")

# Get the generated SQL query
print(f.query)

# Or execute directly and get a DataFrame
df = f.to_dataframe()

A simple configuration with customers and orders generates features like:

• SUM(orders.amount) – Total order value per customer
• COUNT(orders.status|interval=P7D) – Orders in the last 7 days
• ROLLING_MEAN_7(orders.amount) – 7-day moving average of order amounts
• HOLT_WINTERS_TREND_14(orders.amount) – Trend direction over 14 periods

Terminal cockpit

One config against one database, in the terminal: does every source table exist, what will be generated, what has been materialized, and does the manifest still describe the tables on disk. Built on lynkeus, the Textual shell this workspace's data projects share, and installed as an extra because lynkeus needs Python 3.12 (the extra is empty on 3.10 and 3.11, and the verbs say so):

uv sync --extra tui                                  # in a checkout
python -m featurizer tui --config path/to/config.yaml

Tabs 1–5 are the shell's (Status, Runs, Data, Query, Actions); 6–8 are featurizer's: the entity graph as a tree with v running the validator, the feature manifest with / filtering through manifest_matching, and the generated SQL one column group at a time with x for explain analyze (rolled back). Nothing on it writes; featurizer materialize runs from the Actions palette as a subprocess, confirmed first.

The same data prints headlessly for agents and scripts — status, runs list|show, query, actions list|run, each with --json — and two thin verbs over frozen methods came with it: render --config X [--group G] prints query or one entry of query_groups, and materialize --config X --schema S calls to_tables(). Full page: Terminal cockpit.

Output: DataFrame, Arrow, Parquet

to_dataframe() is the notebook/EDA path: it runs the query, returns a pandas DataFrame indexed by (as_of_date, <target id>), and is the right tool for interactive exploration.

For handing a feature matrix to a training pipeline, use the Arrow/Parquet path. It is an optional extra—install with uv sync --extra parquet (pyarrow):

f = Featurizer("config.yaml")
table = f.to_arrow()                 # pyarrow.Table
f.to_parquet("features.parquet")     # writes Parquet directly

These stream the result out of PostgreSQL with binary COPY and decode it column-by-column into Arrow, so the full matrix never round-trips through pandas and two properties hold that to_dataframe does not give cheaply:

• NULL fidelity. A SQL NULL (no qualifying events in the window) stays an Arrow null, never an NaN. pandas coerces integer/boolean columns with nulls to float+NaN, conflating "no data" with "not-a-number".
• Keys as columns. Unlike the DataFrame index, as_of_date and the target id are ordinary leading columns of the table.

This is the beginning of the README. Read the rest at the original. Read the full README ↗