edytlab
Talk to Claude. Get pro-grade audio edits.
A Tauri 2 desktop audio editor where producers, podcasters, and DJs edit audio by chatting with an AI agent. Load two tracks, ask it to bring one to the other's tempo and blend them with a crossfade, and export a WAV. Refine the result through conversation, with an audible A/B at each step.
What it does not do yet. Stem separation is wired in as a tool, but its model inference is not shipped: separate_stems returns an error today (#385). Transcription runs on this computer from v0.8.0, once you download the speech model in Settings (#383, #384). It edits files: no MIDI, no instruments, no plugin hosting. The installers are unsigned (#386).
Local-first, multi-provider, pure-Rust DSP. Builds for macOS (universal), Windows and Linux.
Website and docs
Key features
- Conversational multi-track production. Say "match Neon Rush to Midnight Drive's tempo, start it 8 bars before Midnight Drive ends, crossfade with a low-pass on the outgoing track, then master to −14 LUFS and export a WAV" and the agent plans, executes, and renders. Every edit is a node you can branch from and A/B in the canvas.
- Pure-Rust DSP. Decode (
symphonia), resampling (rubato), and effects written in-house in crates/audio-dsp — a crate with deliberately zero dependencies. DSP quality is non-negotiable and there is no Python in the hot path.
- Local-first. Audio never leaves your machine unless you export. The agent talks to the LLM you pick — a hosted one, or a local model through Ollama; the audio engine runs entirely in-process.
- Multi-provider LLMs out of the box. Six API providers plus your ChatGPT plan: Anthropic, OpenRouter, OpenAI, Groq, Gemini and Ollama — each hosted provider with its own key in the OS keychain (Ollama needs none) — and, as a preview, Continue with ChatGPT to use a ChatGPT Plus or Pro plan's usage instead of a key. Switchable from the Settings panel without reinstall. Adding another is a single
LlmProvider impl.
- Punch in with the rest of the mix playing. Select a region, press Punch, and hear the mix (the track you are replacing is silent inside the region) while you re-record it; the take is placed with the delay measured for your speakers and microphone, or typed by hand, and refused without one, so it does not land late. One history step, Cancel leaves none (#203).
- Branchable session graph. Every state is a node in a DAG. Fork, name, compare, revert — A/B is first-class, not an undo stack. The Graph view draws the DAG next to the timeline.
- Transcription works once the speech model is downloaded; stem separation is not shipped yet.
transcribe runs Whisper (base, about 139 MB, pinned by commit and SHA-256) on the CPU of this computer, on the ONNX Runtime library the installers carry (1.23.2, since v0.7.0). The model is downloaded from Hugging Face (v0.8.0 and later), but only when you click Download in Settings → Account: nothing downloads by itself, and the tool never fetches it; without it transcribe says so and the chat shows a card that sends you to Settings (#383, #384). Your audio never leaves the computer; the words it finds reach your AI provider like any tool result. The transcript drives , , 's mode and the Transcript tab, and a transcript records the track it was made from so an edit by words cuts that track. and 's need no transcript: they key on a voice track's audio (), the voiced sound rather than level. Demucs (stem separation) is wired in as a tool the agent can call, but its model inference is not shipped: returns an error today (); the pieces around it that are written and checked against the reference (htdemucs' STFT and its segment overlap-add) are called by nothing.
Demo videos
Screen recordings of the desktop app with Claude as the agent, each one a DJ job that ends in an exported file. They also play on the website: https://edytlab.com/#demos.
All three are recorded with sound: waits for Claude are sped up, and every playback is in real time.
- Beatmatch and blend two tracks: the DJ asks what they are working with; Neon Rush is time-stretched to Midnight Drive's tempo, keeping its pitch, and started under Midnight Drive's last 8 bars. Midnight Drive fades out under a low-pass filter while Neon Rush fades in; the mix is mastered for streaming and exported as a WAV.
- Extend an intro for mixing: the DJ asks for Solar Flare's tempo and bar length; the drums-only first 8 bars are copied and pasted after themselves, so the intro runs 16 bars; a high-pass filter holds the low end back until the drop, the first 4 bars fade in, and the track is exported as a WAV. You hear the new intro, then the drop.
- A three-track mini-mix: three tracks are brought to one tempo, sequenced with overlaps of about 4 bars and crossfades, matched to about −14 LUFS each and limited at −1 dB on the master, then exported as a WAV.
The list lives in website/lib/demos.ts; its test fails if a listed video or poster is missing from website/public/demos/.
Quick start
Prerequisites:
- Rust toolchain pinned by
rust-toolchain.toml (currently 1.90, installs automatically via rustup)
- Node 22.22.2+ (CI uses 24) and
pnpm 9.15+
- Platform Tauri prerequisites: see https://tauri.app/start/prerequisites/ (Xcode CLT on macOS, MSVC build tools + WebView2 on Windows, the WebKitGTK / GTK / ALSA dev packages on Linux — the exact
apt list is in docs/development-guide.md)
git clone https://github.com/laadtushar/edytlab.git
cd edytlab
pnpm install
pnpm tauri:dev # equivalent to: pnpm --filter @edytlab/desktop tauri dev
The first build is slow — cargo compiles the full Rust workspace plus Tauri. Subsequent runs are incremental.
AI provider setup
edytlab supports six API providers plus your ChatGPT plan out of the box. Each hosted API provider has its own API key in the OS keychain (Ollama needs no key), and you can switch the active provider from the gear icon in the app header without restarting.
Your ChatGPT plan (preview)
Instead of an API key, edytlab can run the assistant on a ChatGPT plan through OpenAI's Sign in with ChatGPT for open-source apps, which OpenAI describes as a preview. Eligible requests then count toward the usage included in the plan, and you set a weekly limit for edytlab in ChatGPT under Settings → Usage. Available for ChatGPT Plus and Pro. It may not be available on workspace plans; that depends on OpenAI and your admin. In Settings → Account choose ChatGPT plan and Continue with ChatGPT; edytlab opens your browser, and you sign in on OpenAI's page. edytlab keeps only a refresh token, in the OS keychain with the API keys (the chatgpt_refresh_token entry of the one secrets item), and a small registration.json (a host ID, the client ID OpenAI issued, the account's identifier and its email as a label) in its config folder. The audio never leaves the computer, as with every provider. Signing out asks OpenAI to revoke the refresh token (and says so if that could not be confirmed), but it does not disconnect edytlab in ChatGPT: do that under ChatGPT settings, Security and login, Sign in with ChatGPT. Details: privacy and the ChatGPT plan section of the user guide. The design is in docs/specs/2026-10-10-chatgpt-subscription-sign-in.md.
Keys are stored via the keyring crate — Keychain on macOS, Credential Manager on Windows, and on Linux the Secret Service (GNOME Keyring, KWallet or KeePassXC, through the crate's sync-secret-service feature, encrypted on the bus with crypto-rust), so they survive a reboot. With no Secret Service running they fall back to the kernel keyring (keyutils) for that boot only, and the app shows a banner saying so; a Save that falls back after launch (the service went away) says so too, in Settings, and holds the dialog open until you go on. Values that earlier builds left in the kernel keyring are moved into the Secret Service on first read. All the secrets are in one keychain item (service app.edytlab.desktop, account secrets), a JSON object from slot name to value: <provider_id>_api_key for each hosted provider and chatgpt_refresh_token. It is read once per process and kept in memory, so macOS asks for your login password at most once per launch instead of once per setting, and a change to a model, a base URL, a reasoning effort or the active provider touches the keychain not at all. Those settings are not secrets: they are in provider-settings.json in the app's config folder (<provider_id>_model, <provider_id>_base_url and <provider_id>_effort per provider, and active_provider), written by replacing the whole file. Builds before v0.9.2 kept every value as an item of its own (anthropic_api_key, openai_model, …); the first launch of a build with this layout reads them once, writes the new item and the file, reads both back, and only then deletes the old items. If any step fails the old items stay and are used as before, and the next launch tries again (#734, ADR 0009).
The model picker is a combo (free-form input + curated suggestions from the live catalogue) so a brand-new model id works the moment you know it.
Ollama is sent a smaller set of tools than the hosted providers: the common edits plus any tool your message names, with shortened descriptions, so a first request fits a small local context (see Request size and small-context models). A tool that is not offered still runs if the model calls it.
Architecture
┌──────────────────────────────────────────────────────────────────┐
│ Tauri shell (React 19 + Vite + Tailwind) │
│ Canvas (waveform) · Chat panel · Graph view · Settings │
└──────────────────────────────────────────────────────────────────┘
│ tauri::command + Tauri events
┌──────────────────────────────────────────────────────────────────┐
│ Rust core (cargo workspace) │
│ crates/ai — LlmProvider trait, agent loop, keychain │
│ crates/tools — 99 deterministic tools + dispatcher │
│ crates/session — DAG of session states, fork/diff/compare │
│ crates/audio-* — decode, DSP, engine, time-domain ops │
│ crates/ml-* — Whisper, diarize (run); Demucs (stub) │
└──────────────────────────────────────────────────────────────────┘
│
┌──────────────────────────────────────────────────────────────────┐
│ LLM providers (pluggable at runtime) │
│ Anthropic · OpenRouter · OpenAI · Groq · Gemini · Ollama │
│ (six API providers plus your ChatGPT plan) │
└──────────────────────────────────────────────────────────────────┘
Single user turn: chat message → crates/ai agent loop (reading the provider's SSE stream) → tool calls dispatched in crates/tools → DSP in the audio engine → Tauri events (agent://…, emitted from apps/desktop/src-tauri/src/events.rs) stream back to the chat and timeline. The LlmProvider trait in crates/ai/src/provider.rs owns auth, endpoint path, request serialization, and stream parsing — that's the extension point for new providers.
For the long version, read docs/specs/2026-05-05-conversational-audio-editor-design.md §5.
Tech stack
- Shell: Tauri 2, Rust workspace (edition 2021, toolchain 1.90),
cargo profile-release with lto = true
- Frontend: React 19, Vite 7, Tailwind 4,
@xyflow/react for the graph view, wavesurfer.js for waveforms
- Audio:
cpal (microphone capture), symphonia (decode), rubato (resampling), realfft (spectral), hound (WAV), flac-codec and rusty_mp3 (export), plus in-house effects in crates/audio-dsp
- ML:
ort (ONNX Runtime, load-dynamic) for Demucs / Whisper — the installers carry the runtime library (1.23.2, since v0.7.0), and ml-whisper runs Whisper on a model the user downloads (transcribe), but no model is bundled and stem separation does not run in the app yet (#383)
- LLM:
reqwest + eventsource-stream for SSE; keyring for OS-keychain credential storage
- Test:
cargo test (unit + integration), vitest (frontend), Playwright (apps/desktop/e2e/, the frontend in Chromium with the IPC boundary faked), wiremock for HTTP fakes
- Tooling: pnpm 9 workspace, conventional commits
Project structure
apps/
desktop/ Tauri shell — React frontend (src/) + Rust bridge (src-tauri/)
cli/ Headless one-turn agent driver (edytlab-cli) for E2E tests and smoke runs
crates/
ai/ LLM provider abstraction, agent loop, keychain, prompt cache
tools/ 99 audio-editing tools (load, cut, gain, normalize, render, …)
session/ Session-graph data model, DAG storage, fork/diff/compare
audio-decoder/ File decode (symphonia)
audio-dsp/ Sample-level DSP shared by the tools and the renderer (no dependencies)
audio-engine/ DSP graph + render
audio-time/ Time-stretch and pitch-shift primitives
audio-analysis/ BPM, key, beat-grid, transients
recorder/ Microphone capture to WAV; recording-latency measurement; the punch-in's monitor and take (cpal)
memory/ Global and per-project markdown memory for the system prompt
skills/ User skill library, appended to the prompt when a skill matches
agent_profiles/ Saved model + tool-whitelist + prompt profiles
mcp/ MCP server config, lifecycle and JSON-RPC
ml-demucs/ Stem separation (inference is a stub, #385)
ml-diarize/ Speaker diarization (pyannote-style pipeline on ONNX; the `diarize` tool runs it on the speaker model the user downloads in Settings, #168)
ml-whisper/ Transcription (run by the `transcribe` tool on the model the user downloaded, #384)
ml-pipeline/ Shared ONNX runtime gate, verified model downloader + the user-started speech-model download (#383) + inference cache
docs/ Specs, handover, packaging notes
.github/workflows/ CI, cargo audit, auto-release, unsigned dev/release and signed release pipelines
tests/ Shared fixture notes and golden-data placeholders
Development
The full acceptance gate (mirrors CLAUDE.md) — run before opening a PR:
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
pnpm --filter @edytlab/desktop test
pnpm --filter @edytlab/desktop test:slow-scheduler # same suite, React's scheduler 40 ms late
pnpm --filter @edytlab/desktop typecheck # tsc -b: app, tests and vite.config.ts
pnpm --filter @edytlab/desktop typecheck:e2e
pnpm --filter @edytlab/desktop test:e2e # Playwright in Chromium; needs `playwright install chromium` once
Common one-shots:
pnpm tauri:dev # run the desktop app in dev mode
pnpm --filter @edytlab/desktop test:watch # vitest in watch mode
cargo test -p ai # test a single crate
cargo run -p edytlab-cli -- --message "<text>" # one agent turn, headless (key from ANTHROPIC_API_KEY)
The Tauri bundle build is intentionally not in CI (too slow); release workflows cover that path.
Releases
- Auto dev releases.
auto-release.yml listens to ci.yml's workflow_run on main. On green, it tags v<version>-dev.<run_number> and dispatches release-dev.yml, which builds unsigned bundles — a universal macOS .dmg, Windows .msi + NSIS .exe, Linux .deb + AppImage — and attaches them to a GitHub Release. On a green matrix the publish job flips that release out of draft, so every green push to main publishes a dev prerelease; it is marked prerelease with make_latest=false, so it never takes the Latest badge from a real release. A draft is what a failed matrix leaves behind.
- The ONNX Runtime in every installer. Each release leg fetches the pinned ONNX Runtime (
apps/desktop/src-tauri/ml-runtime.json), builds with its platform's tauri.ml-runtime.<os>.conf.json overlay, and checks every installer before uploading it (scripts/installer-ml-runtime.py): the library and its notices are where the app looks, and the app binary from that installer reports it bundled (--ml-runtime-status). release-dev.yml dispatched with dry_run: true builds and checks without releasing anything.
- Versioned releases (unsigned for now).
release-dev.yml dispatched with channel: release publishes a real release — not a prerelease, marked Latest. v0.2.0 to v0.9.3 shipped this way.
- Signed releases.
release-signed.yml builds all three platforms in one matrix — Apple notarization on mac, Authenticode + DigiCert timestamp on Windows, unsigned .deb/AppImage on Linux. Manual workflow_dispatch, gated on signing secrets being provisioned. They are not provisioned yet, so every release so far is unsigned (#386). It builds first and uploads only after signing, so the published assets are always the signed bytes. See docs/packaging-windows.md for the Windows side, including the SmartScreen reputation note.
- Bundle targets are pinned in
tauri.conf.json to ["app", "dmg", "msi", "nsis", "deb", "appimage"] — don't revert to .
App version is canonical in apps/desktop/src-tauri/tauri.conf.json; the package.json files and the Cargo workspace's [workspace.package] version mirror it, and apps/desktop/src/__tests__/appVersion.test.ts fails if they drift.
Contributing
Working-style rules, branch naming, conventional-commit prefixes, and acceptance gates are codified in CLAUDE.md. One concern per PR; open as draft, squash-merge once CI is green. The contributor guide is docs/contributing.md (how to add a tool, provider or component, test requirements); there is no separate CONTRIBUTING file.
Roadmap
v1 (in progress) ships in three phases — single-track edit, mashup, conversational mix engineer. Spec §9 has the breakdown.
Out-of-scope for v1, on the post-v1 roadmap:
- v2 — backlog. Real-time DJ performance mode (live decks, controllers, beat-jump). Note-level / RipX-style harmonic editing. VST3 / CLAP plugin hosting. DAW round-trip (OMF / AAF / Logic project export).
- Deferred. ASIO support on Windows. Mobile companion. Multi-user collaboration. Cloud project sync.
- Out. Music generation (Suno / Udio territory) — edytlab edits and mixes existing audio.
Configuration
- API keys: stored in the OS keychain via the
keyring crate, all in one item (secrets, a JSON map with a <provider>_api_key entry per provider). Never committed; never logged. The ChatGPT plan has no key: its sign-in's refresh token is the chatgpt_refresh_token entry of the same item, and its non-secret registration is chatgpt/registration.json in the app config folder.
- Active provider + model: stored in
provider-settings.json in the app config folder (active_provider, and per provider model, plus base_url and effort when set; no secret is ever in it) and mirrored in localStorage (edytlab.provider, edytlab.model.<provider>).
- Rust toolchain: pinned via
rust-toolchain.toml. CI uses the same pin.
- Tauri config:
apps/desktop/src-tauri/tauri.conf.json — bundle targets, identifier, signing entitlements, WebView2 install mode.
License
MIT — see LICENSE. The libraries, fonts and code edytlab is built from have licences of their own; their texts and copyright notices are in THIRD_PARTY_NOTICES.txt, which every installer carries and the app shows under Settings → Account → Open source licences.
