duck soup。
📖 Full user guide: https://henrik716.github.io/duck-soup/
duck soup makes config-driven geodata ETL on DuckDB as easy as... well, duck soup!
It's a lightweight, lightning-fast replacement for building heavy workspaces in tools like FME (Safe Software Feature Manipulation Engine), or writing custom, error-prone Python scripts. Describe your pipeline as a simple YAML file — sources, a base, join/geoprocessing steps, attribute mapping, one or more output layers — or build it visually in the interactive web editor.
Under the hood, everything runs inside DuckDB using the powerful spatial extension: joins, geoprocessing (buffers, clips, overlays, dissolves, ...), and field mappings all compile down to a chain of SQL views, written straight to an output GeoPackage or GeoParquet. A single YAML file can define several independent pipelines, each fanning out to multiple layers, all written into one shared output.
Run it
Install the duck-soup command with uv (recommended) or pipx. Both give it its own
isolated environment and need no admin rights:
# uv: works with or without Python installed; it downloads a matching one itself
uv tool install duck-soup-etl
# pipx: if you already have Python 3.11+ and prefer pipx
pipx install duck-soup-etl
Then start the editor and open http://localhost:8000:
duck-soup serve
Internet access is needed on first run so DuckDB can download its spatial extension.
Don't have uv or pipx yet?
# uv, on Windows (PowerShell)
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
# uv, on macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
# pipx
python -m pip install --user pipx
python -m pipx ensurepath # then open a new terminal
To upgrade: uv tool upgrade duck-soup-etl or pipx upgrade duck-soup-etl.
If you used plain pip install duck-soup-etl and duck-soup isn't found, that's a
PATH issue, not a broken install. python -m duck_soup.cli serve always works.
Docker
For servers, scheduled jobs, or machines where you can't install Python:
docker run -p 8000:8000 -v "$PWD":/data ghcr.io/henrik716/duck-soup
This mounts the current directory as /data (also the container's working directory),
so pipelines/, data/ and output/ are read from and written to it. Folders outside
it, such as another drive or a network share, need their own -v mount. To upgrade:
docker pull ghcr.io/henrik716/duck-soup.
Try it
duck-soup tutorial copies a small sample town (Pondsworth) and a config with an
example of every step and mapping option into a folder:
mkdir -p ~/duck-soup && cd ~/duck-soup # the editor's default project folder
duck-soup tutorial
duck-soup serve # then pick "pondsworth" in the editor's load list
The tutorials walk through it.
Command line
| Command | What it does |
|---|
duck-soup check <file> | validate a pipeline YAML without reading any data |
duck-soup run <file> | run every pipeline in the file and write the output |
duck-soup serve [--host] [--port] | start the web editor (default 127.0.0.1:8000); DUCK_SOUP_ROOT sets the project folder (default ~/duck-soup) |
duck-soup tutorial [folder] [--force] | copy the tutorial data and config into a project folder |
check and run exit non-zero on any failure, so they slot into scripts and CI.
Relative paths in a pipeline resolve against the folder you run the command from.
The editor has no authentication and can read files and run SQL on the machine it runs
on. Keep it on 127.0.0.1 or behind something that controls access.
Scheduling
A run is one command, so cron, systemd timers, Windows Task Scheduler, Docker and CI can
all schedule it:
# crontab: rebuild every night at 02:00
0 2 * * * cd /srv/gis && /home/gis/.local/bin/duck-soup run pipelines/nightly.yaml >> logs/nightly.log 2>&1
With overwrite: true the old output is deleted before the sources are read, so a
failed run leaves no file. The scheduling guide
has a recipe per scheduler and shows how to swap in the new file only after a good run.
This is the beginning of the README. Read the rest at the original. Read the full README ↗
