xrtoolz

Composable operators for geoprocessing Earth System Data Cubes.

xrtoolz gives every preprocessing, physics, inference, and evaluation step a single shape — an Operator: a callable that maps xarray in to xarray out, carries its configuration, and reports its shape signature. Operators chain linearly via Sequential or wire into a DAG via the functional Graph API — the same operator works in both.

The carrier-agnostic composition core lives in pipekit; xrtoolz adds the xarray-specific operator families on top.

📖 Documentation · 🧩 API Reference · 🧪 Tutorials

Quick start

from pipekit import Sequential
from xrops.anomaly import RemoveMean
from xrops.ocn import Streamfunction, GeostrophicVelocities
from xrops.viz.validation import SpatialMapPanel

# Geostrophic surface currents from sea-surface height, as one pipeline
pipeline = Sequential(
    RemoveMean(dim=("lat", "lon")),  # subtract the spatial mean
    Streamfunction(),            # ψ = g·η / f
    GeostrophicVelocities(),     # (u_g, v_g) from ψ
)

currents = pipeline(ds)          # xr.Dataset in → xr.Dataset out
fig = SpatialMapPanel(variable="u")(currents)

Need branching or shared intermediates? Wire the same operators as a DAG with Graph(inputs=…, outputs=…).

Architecture — three layers

Layer	Name	What it is
0	Primitives	Pure functions (xr.Dataset, …) → xr.Dataset. Backed by jaxtyped numpy kernels.
1	Operators	Operator subclasses with a uniform __call__ / get_config / __repr__; compose with Sequential.
2	Graph	Input / Node / Graph DAG API and ModelOp inference wrappers.

Drop to Layer 0 when you want the raw function; stay at Layer 1–2 to compose. Operator, Sequential, Graph, Input, Node, Tap are re-exported from pipekit at the top level.

Module map

Module	Scope
geom, mask, anomaly, extremes, wavelets, cf	Generic geoprocessing — validation, subsetting, regions, masks, CRS, climatology, wavelets, extremes
calc	Finite-difference gradient / divergence / curl / laplacian
transforms	Fourier / wavelet / DCT spectra, spectral fluxes, decompositions, morphology
interpolate	Regrid, coarsen, gap-fill, bin, smooth, point-sample
ocn	Ocean physics — geostrophy, vorticity, KE, stratification, SSH
atm · rs	Atmospheric & remote-sensing physics (reserved; planned)
metrics	Pixel, spectral, multiscale, physical, masked, distributional evaluation
budgets	Conservation-budget residuals (heat, salt, volume, KE)
learn	scikit-learn on DataArrays, Datasets and DataTrees, and the .sklearn accessor

Data acquisition (CMEMS / CDS / AEMET archive readers) lives in the xrtoolz-products workspace member, which imports as xrproducts.

Install

uv add xrtoolz

The repository is a uv workspace; the installable distributions live under packages/* (the repo root itself ships no code). xrtoolz depends on its two workspace siblings plus pipekit, pipekit-array and geotoolz-ops — all pre-PyPI — so a Git install must name each of them explicitly:

uv pip install \
  "xrtoolz @ git+https://github.com/jejjohnson/xrtoolz@main#subdirectory=packages/xrtoolz" \
  "xrtoolz-ops @ git+https://github.com/jejjohnson/xrtoolz@main#subdirectory=packages/xrtoolz-ops" \
  "xrtoolz-products @ git+https://github.com/jejjohnson/xrtoolz@main#subdirectory=packages/xrtoolz-products" \
  "pipekit @ git+https://github.com/jejjohnson/pipekit@main#subdirectory=packages/pipekit" \
  "pipekit-array @ git+https://github.com/jejjohnson/pipekit@main#subdirectory=packages/pipekit-array" \
  "geotoolz-ops @ git+https://github.com/jejjohnson/geotoolz@main#subdirectory=packages/geotoolz-ops" \
  "geotoolz-array @ git+https://github.com/jejjohnson/geotoolz@main#subdirectory=packages/geotoolz-array"

For development, clone and let the workspace resolve everything instead (see below). A scoped package installs on its own — e.g. "xrtoolz-products @ git+https://github.com/jejjohnson/xrtoolz@main#subdirectory=packages/xrtoolz-products" needs nothing else from the stack.

From source

git clone https://github.com/jejjohnson/xrtoolz.git
cd xrtoolz
make install      # install all dependency groups (resolves pipekit from GitHub)
make test         # run the test suite
make docs-serve   # preview the docs locally

Status

Pre-alpha. The API is stabilising domain by domain — see the full design documentation for motivation, architecture, boundaries, conventions, and roadmap.

License

MIT. See LICENSE.