drt — data reverse tool

The reverse leg of your data stack.

dlt loads data in, dbt transforms it, and drt activates it back out — reverse ETL from your warehouse to the tools your team works in, as YAML in your own repo. No dashboard to babysit, no audience builder duplicating what dbt already does, no hosted runtime billing you per row. Your data goes straight from your warehouse to the destination — never through a drt-hosted intermediary.

pip install drt-core          # core (DuckDB included)
drt init && drt run

---

Why drt?

Same DX as dbt. If you know dbt run, you already know drt run — declarative YAML, versioned in Git, reviewed in PRs.

CI-native. Exit codes and --output json drop straight into GitHub Actions, cron, or Dagster / Airflow / Prefect — no GUI, no clickops.

LLM-native. A built-in MCP server and Claude Code skills let AI tools author and run your syncs.

Free and open source. Every connector, the CLI, the MCP server, and the sync engine — Apache 2.0, self-hosted, no lock-in.

Review before you write. drt plan shows the exact rows a sync would create, update or delete; drt apply writes only if nothing changed since you looked, and guards stop a runaway delete.

Fill, don't overwrite. write_policy: fill_empty writes a column only where the destination is empty, so values people already entered stay.

What's always free? All connectors, CLI, MCP server, and sync engine. See OPEN_CORE.md for the open core boundary.

---

Quickstart

No cloud accounts needed — DuckDB + httpbin.org, three commands.

pip install drt-core
mkdir my-drt-project && cd my-drt-project
drt init --template duckdb_to_rest

That scaffolds a runnable syncs/duckdb_to_rest.yml. Seed a tiny DuckDB table and run:

python -c "
import duckdb
c = duckdb.connect('warehouse.duckdb')
c.execute('''CREATE TABLE IF NOT EXISTS users AS SELECT * FROM (VALUES
  (1, 'Alice', '[email protected]'),
  (2, 'Bob',   '[email protected]'),
  (3, 'Carol', '[email protected]')
) t(id, name, email)''')
c.close()
"
drt run --dry-run   # preview, no data sent
drt run             # POST each row to httpbin.org
drt status          # check results

Other starter templates

drt init --template list             # see all available templates
drt init --template postgres_to_slack
drt init --template duckdb_to_hubspot

Each template prints next-steps for the env vars / source data it needs. See examples/ for the full collection (Discord, Google Sheets, GitHub Actions, MySQL, ClickHouse, BigQuery, …) and docs/connectors/ for per-connector reference.

Customizing your sync

For a guided wizard that walks you through profile + project setup:

drt init   # interactive — picks a source, configures profile, scaffolds project

Both flows produce the same project shape (drt_project.yml, syncs/, .drt/). drt sources --detailed and drt destinations --detailed print every connector's required env vars and a sample YAML stanza — useful when hand-authoring beyond the templates.

Existing metadata can seed the same sync files: drt init --from-dbt target/manifest.json interactively scaffolds dbt models, while drt init --from-ossie semantic_model.yaml [--dataset NAME ...] [--destination postgres|hubspot] scaffolds keyed Apache Ossie datasets. Review the generated TODOs, then run drt validate.

---

Review before you write

drt plan is a read-only, key-level change set; drt apply re-computes it and refuses to write if the data, the config or the plan itself changed since you reviewed it.

drt plan fill_industry --out plan.json   # nothing is written
drt apply plan.json                      # writes only what you reviewed

In the demo above (Docker + pip, no cloud accounts), a CRM table already has an industry filled in by reps for about 70% of companies. The sync uses write_policy: fill_empty: the plan says 450 rows to update, 300 rep-entered values kept, and apply fills only the blanks. A deliberately bad variant (a 25-row mirror into the whole table) proposes 1,487 deletes; a max_delete_pct: 5 guard stops it at plan time and apply refuses.

• Try it: examples/review-demo — one script, a 60-second walkthrough, with copy-paste prompts for an AI agent (drt_plan → human approval → drt_apply).
• In CI: drt deploy github-actions --with-plan scaffolds a plan comment on the pull request and an apply on merge.
• Read the guide: plan / apply, including what it guarantees and what it does not (it is a guard in front of a normal run, not a transaction; plan needs a destination drt can read back, which today means SQL databases and warehouses).

CLI Reference

drt init                    # initialize project
drt init --from-dbt target/manifest.json  # scaffold selected dbt models
drt init --from-ossie model.yaml          # scaffold keyed Apache Ossie datasets
drt list                    # list sync definitions
drt sources                 # list available source connectors
drt destinations            # list available destination connectors
drt run                     # run all syncs
drt run --select <name>     # run a specific sync (globs work: 'users_*')
drt run --select tag:<tag>  # run syncs matching a tag (repeat --select to union)
drt run --select destination:<type>  # run syncs by destination type
drt run --exclude <name>    # subtract syncs (same grammar as --select)
drt run --failed            # re-run only syncs that failed last time
drt run --limit 10          # sampled run: send only N rows (watermark frozen)
drt run --fail-fast         # stop scheduling after the first failure
drt run --vars 'lookback_days: 1'   # override project vars: for this run
drt run --threads 4         # parallel sync execution
drt run --dry-run           # dry run
drt run --verbose           # show row-level error details
drt run --output json       # structured JSON output for CI/scripting
drt run --log-format json   # structured JSON logging to stderr
drt run --profile prd       # override profile (or DRT_PROFILE env var)
drt run --cursor-value '…'  # override watermark cursor for backfill
drt run --force-guards      # explicitly override a tripped run-time delete guard
drt test                    # run post-sync validation tests
drt test --select <name>    # test a specific sync
drt test --store-failures   # sample up to N failing rows per failed test (masked)
drt build                   # run each sync and its tests in one pass
drt validate                # validate sync YAML configs
drt plan <sync> --out plan.json   # save a reviewable change set (read-only)
drt apply plan.json         # apply it only if the world still matches
drt apply plan.json --force-guards  # explicitly override reviewed guard trips
drt deploy github-actions --with-plan   # PR comment with the plan; apply on merge
drt status                  # show recent sync status
drt status --output json    # JSON output for status
drt profile list            # list credential profiles in ~/.drt/profiles.yml
drt profile show <name>     # show a profile (secrets masked)
drt profile test <name>     # verify a profile's source connectivity
drt profile add <name>      # interactively add a profile
drt profile remove <name>   # remove a profile
drt serve                   # start HTTP webhook endpoint
drt docs generate --format mermaid  # print project DAG as Mermaid
drt docs generate --format dbt-exposures  # print dbt exposure YAML for ref() syncs
drt deploy github-actions   # scaffold a scheduled sync workflow (drt-action + secrets wired)
drt deploy cloud-run-job --schedule "40 3 * * *"  # Dockerfile + Cloud Run/Scheduler specs
drt deploy k8s-cronjob --schedule "40 3 * * *"    # hardened batch/v1 CronJob manifest
drt mcp run                 # start MCP server (requires drt-core[mcp])
drt --install-completion    # install shell completion (bash/zsh/fish)
drt --show-completion       # show completion script

See the scheduling decision guide before choosing between warehouse-native scheduling, Dagster sensors, drt serve, or a deploy scaffold.

Visualize your syncs

Generate a Mermaid DAG from your local drt_project.yml and syncs/*.yml files:

drt docs generate --format mermaid > dag.md

graph LR
    subgraph Sources
        src_bigquery_prod["bigquery_prod<br/><i>bigquery</i>"]
    end
    subgraph Syncs
        sync_users_to_hubspot{{"users_to_hubspot<br/><i>upsert</i>"}}
        sync_accounts_to_hubspot{{"accounts_to_hubspot<br/><i>upsert</i>"}}
    end
    subgraph Destinations
        dst_hubspot_contacts["hubspot (contacts)<br/><i>hubspot</i>"]
    end
    src_bigquery_prod -->|extract| sync_users_to_hubspot
    src_bigquery_prod -->|extract| sync_accounts_to_hubspot
    sync_users_to_hubspot -->|load| dst_hubspot_contacts
    sync_accounts_to_hubspot -->|load| dst_hubspot_contacts
    sync_users_to_hubspot -.lookup.-> sync_accounts_to_hubspot

Shell completion

Shell completion is supported for bash, zsh, and fish:

# Recommended: auto-install for your current shell (idempotent)
drt --install-completion

# Or manually add to your shell config (run once from the target shell)
drt --show-completion >> ~/.bashrc   # bash
drt --show-completion >> ~/.zshrc    # zsh
drt --show-completion > ~/.config/fish/completions/drt.fish  # fish

Note: --show-completion outputs the script for your current shell. Run it from the shell you want to configure. The manual >> append is not idempotent — run it once only.

After installation, restart your shell and tab-complete commands and options.

---

MCP Server

Connect drt to Claude, Cursor, or any MCP-compatible client so you can run syncs, check status, and validate configs without leaving your AI environment.

pip install drt-core[mcp]
drt mcp run

Claude Desktop (~/Library/Application Support/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "drt": {
      "command": "drt",
      "args": ["mcp", "run"]
    }
  }
}

Available MCP tools:

Tool	What it does
drt_list_syncs	List all sync definitions
drt_run_sync	Run a sync (dry_run, compute_diff; forced guards require approved_by)
drt_run_test	Run post-sync validation tests (mirrors drt test)
drt_get_status	Get last run result(s)
drt_get_history	Get recent sync run history
drt_validate	Validate sync YAML configs
drt_get_schema	Return JSON Schema for config files
drt_list_connectors	List available sources and destinations
drt_dlq	Inspect a sync's Dead Letter Queue (depth + failed records)
drt_retry	Replay a sync's Dead Letter Queue (mirrors drt retry)
drt_get_manifest	Machine-readable sync catalog + lineage (drt docs JSON)
drt_list_profiles	List credential profiles (name + type, no secrets)
drt_test_profile	Connectivity check for a credential profile
drt_doctor	Environment diagnostics (mirrors drt doctor)
drt_state_show	Stored watermark + last-run state
drt_state_reset	Reset watermark / run state / tracked-mirror keys
drt_plan	Read-only change set for a sync + a plan_id (mirrors drt plan)
drt_apply	Apply a fetched plan; needs approved_by, with optional explicit force_guards

---

AI Skills for Claude Code

Install the official Claude Code skills to generate YAML, debug failures, and migrate from other tools — all from the chat interface.

Install via Plugin Marketplace (recommended)

/plugin marketplace add drt-hub/drt
/plugin install drt@drt-hub

Tip: Enable auto-update so you always get the latest skills when drt is updated: /plugin → Marketplaces → drt-hub → Enable auto-update

Manual install (slash commands)

Copy the files from .claude/commands/ into your drt project's .claude/commands/ directory.

Skill	Trigger	What it does
/drt-create-sync	"create a sync"	Generates valid sync YAML from your intent
/drt-debug	"sync failed"	Diagnoses a specific error and suggests fixes
/drt-review-sync	"what will this change?"	Plans a sync, explains it, applies only after human approval
/drt-troubleshoot	"drt isn't working"	Walks a full top-to-bottom diagnostic checklist
/drt-init	"set up drt"	Guides through project initialization
/drt-migrate	"migrate from Census"	Converts existing configs to drt YAML

---

Connectors

Per-connector reference: docs/connectors/ · Discoverable from the CLI: drt sources --detailed / drt destinations --detailed

Sources

Connector	Status	Install	Auth
BigQuery	✅ v0.1	pip install drt-core[bigquery]	Application Default / Service Account Keyfile
DuckDB	✅ v0.1	(core)	File path
PostgreSQL	✅ v0.1	pip install drt-core[postgres]	Password (env var)
Snowflake	✅ v0.5	pip install drt-core[snowflake]	Password (env var)
SQLite	✅ v0.4.2	(core)	File path
Redshift	✅ v0.3.4	pip install drt-core[redshift]	Password (env var)
ClickHouse	✅ v0.4.3	pip install drt-core[clickhouse]	Password (env var)
MySQL	✅ v0.5	pip install drt-core[mysql]	Password (env var)
Databricks	✅ v0.6	pip install drt-core[databricks]	Access Token (env var)
Delta Lake	✅ v0.8	pip install drt-core[deltalake]	Storage options (env var)
Iceberg	✅ v0.8	pip install drt-core[iceberg]	Catalog properties (env var)
SQL Server	✅ v0.6	pip install drt-core[sqlserver]	Password (env var)
REST API	✅ v0.7	(core)	Bearer / API Key / Basic / OAuth2

Destinations

Connector	Status	Install	Auth
REST API	✅ v0.1	(core)	Bearer / API Key / Basic / OAuth2
Slack Incoming Webhook	✅ v0.1	(core)	Webhook URL
Discord Webhook	✅ v0.4.2	(core)	Webhook URL
GitHub Actions	✅ v0.1	(core)	Token (env var)
HubSpot	✅ v0.1	(core)	Token (env var)
Zendesk	✅ v0.7	(core)	Basic (email + API token)
Amplitude	✅ v0.7	(core)	Project API key (env var)
Klaviyo	✅ v0.8	(core)	Private API key (env var)
Mixpanel	✅ v0.8	(core)	Project token / service account
Google Ads	✅ v0.6	(core)	OAuth2 Client Credentials
Meta Conversions	✅ next	(core)	Long-lived access token (env var)
Google Sheets	✅ v0.4	pip install drt-core[sheets]	Service Account Keyfile
PostgreSQL (upsert)	✅ v0.4	pip install drt-core[postgres]	Password (env var)
MySQL (upsert)	✅ v0.4	pip install drt-core[mysql]	Password (env var)
ClickHouse	✅ v0.5	pip install drt-core[clickhouse]	Password (env var)
Parquet file	✅ v0.5	pip install drt-core[parquet]	File path
Amazon S3	✅ v0.7.9	pip install drt-core[s3]	AWS credential chain / env vars
Amazon SQS	✅ next	pip install drt-core[sqs]	AWS credential chain / env vars

Integrations

Connector	Status	Install
Dagster	✅ v0.4	pip install dagster-drt
Prefect	✅ v0.6	(core)
Airflow	✅ v0.6	(core)
dbt manifest reader	✅ v0.4	(core)

---

Roadmap

Upcoming releases → ROADMAP.md (scope, themes, targets) Issue-level tracking → GitHub Milestones Looking to contribute? → Good First Issues

Shipped: now on v1.1.0 — Warehouse parity: managed tables, warehouse-backed state/history/DLQ, diff-based incremental and mirror.strategy: diff now run on Snowflake, Databricks and BigQuery as well as Postgres, each verified against a live warehouse. Builds on v1.0.0, the first stable release (frozen Source/Destination/StateManager Protocols, VERSIONING.md deprecation policy). No breaking changes.

Milestone	Highlights
v0.1–0.3	Core CLI · BigQuery / DuckDB / Postgres sources · REST / Slack / HubSpot destinations · incremental sync · MCP server + AI Skills for Claude Code
v0.4–0.6	20+ destinations (Sheets, SQL upserts, Notion, Twilio, Salesforce…) · dagster-drt / Airflow / Prefect · drt test · parallel --threads · JSON Schema validation
v0.7 ✅	Production Ready — graceful shutdown · sync history · atomic table swaps · drt doctor · opt-in telemetry · sync.mode: mirror · DLQ + drt profile
v0.7.9–0.7.11 ✅	Cloud destinations (S3 / GCS / Azure / Databricks / BigQuery / Elasticsearch) · Delta Lake + Iceberg sources · tracked & scoped mirror · PII masking · VS Code extension
v0.8.0 ✅	drt docs lineage site (safe-by-default labels, static SVG DAG, byte-identical) · project vars: · drt build · --fail-fast / --limit / --failed · selection v2 · incremental REST API · watermark.lag · drt deploy github-actions
v0.8.1 ✅	sync.match_policy update_only / create_only (Postgres + HubSpot) · drt docs generate --inline (self-contained HTML for object-store hosting) · manifest schema v2 (run history · column facts · DLQ depth) · tracked-mirror no-CREATE-grant · Retry-After
v0.8.2 ✅	drt docs --inline → single navigable HTML object — the whole catalog in one file with in-page nav, so it renders and navigates on an authenticated GCS / S3 object URL (completes v0.8.1's --inline)
v0.8.3 ✅	--dry-run --diff previews mirror DELETEs (tracked / destination / scoped, read-only, labelled apart from a replace rebuild) · keyed destination lookup instead of a full scan · alerts.on_degraded (row-error %, duration SLA, empty source, DLQ depth) · drt test custom SQL tests, ,

Next: v1.3 Depth and durability → v1.x Rust Engine

---

Orchestration: dagster-drt

Community-maintained Dagster integration. Expose drt syncs as Dagster assets with full observability.

pip install dagster-drt

from dagster import AssetExecutionContext, Definitions
from dagster_drt import drt_assets, DagsterDrtResource

@drt_assets(project_dir="path/to/drt-project")
def my_syncs(context: AssetExecutionContext, drt: DagsterDrtResource):
    yield from drt.run(context=context)

defs = Definitions(
    assets=[my_syncs],
    resources={"drt": DagsterDrtResource(project_dir="path/to/drt-project")},
)

For event-driven activation, build_drt_change_sensor() watches metadata-only change signals from Delta Lake, Iceberg, Snowflake, or SQL Server. The same resource also runs explicitly selected syncs from a plain Dagster @op, returns a chainable DrtEventIterator for source row-count checks, and powers the declarative DrtSyncComponent for defs.yaml projects.

See dagster-drt README for full API docs (sensors, Components, @op, Translator, Pipes support, and dry-run config).

---

CI/CD: GitHub Action

Run drt syncs straight from CI/CD with the official drt-hub/drt-action — no infrastructure, just a few lines of YAML. Trigger on a schedule, on every push, or right after dbt finishes.

- uses: drt-hub/drt-action@v1
  with:
    select: '*'
    extras: postgres
  env:
    PG_PASSWORD: ${{ secrets.PG_PASSWORD }}

Inputs cover sync selection (select), connector extras, profile, dry-run and threads; outputs expose status, succeeded, failed and duration-seconds (plus a step-summary table). Secrets are passed via env: and resolved by drt's *_env keys. See the action README for the secrets pattern and more examples (run-after-dbt, PR preview).

---

Ecosystem

drt is designed to work alongside, not against, the modern data stack:

---

Telemetry

drt collects no telemetry by default. Opting in helps us understand which sources / destinations / sync modes are actually used, so we can prioritise.

drt config set telemetry.enabled true     # opt in
drt config show-telemetry                 # preview the exact payload that would be sent
drt config set telemetry.enabled false    # opt out
DO_NOT_TRACK=1 drt run                    # universal kill switch — overrides everything

When opted in, drt sends one sync_completed event per sync. The only properties we collect are these 9 fields: drt_version, python_version, os, source_type, destination_type, sync_mode, rows_synced, duration_seconds, status. The wire envelope additionally carries event, distinct_id (a per-machine random UUID at ~/.drt/.anonymous_id), timestamp, and api_key. Sync names, model SQL, destination URLs, credentials, and project paths are never transmitted — the allow-list is enforced at the function-signature level in drt/telemetry.py. By default events go to PostHog Cloud (EU region); override with DRT_TELEMETRY_ENDPOINT and DRT_TELEMETRY_API_KEY for self-hosted PostHog or a custom collector.

Note: drt itself never transmits your IP, but the receiving PostHog backend records the TCP source IP as $ip. See docs/telemetry.md for details and how to disable / substitute the backend.

For full details see docs/telemetry.md.

Contributing

We welcome contributions of all sizes — from typo fixes to new connectors. drt has a transparent contributor ladder so your work builds toward greater trust and responsibility over time.

• Get started: CONTRIBUTING.md — setup, workflow, and your first connector tutorial
• Pick something to work on: Good First Issues
• Understand how decisions are made: GOVERNANCE.md
• What's free vs. enterprise: OPEN_CORE.md
• Versioning & breaking changes: VERSIONING.md

Contributors ✨

Every contribution — code, docs, bug reports, ideas — makes drt better. Thank you 💜

Sponsors

drt is free and open source. If it saves your team time, sponsoring drt-hub keeps it maintained: it pays for the live warehouse accounts every release is tested against. See SPONSORS.md for how it works and who is backing it.

Disclaimer

drt is an independent open-source project and is not affiliated with, endorsed by, or sponsored by dbt Labs, dlt-hub, or any other company.

"dbt" is a registered trademark of dbt Labs, Inc. "dlt" is a project maintained by dlt-hub.

drt is designed to complement these tools as part of the modern data stack, but is a separate project with its own codebase and maintainers.

License

Apache 2.0 — see LICENSE.