VoxCtrl

A high-performance, private, on-device voice-to-text dictation application and programmable voice input broker built natively with Rust, Tauri 2, and Svelte 5.

Zero Telemetry • Zero Cloud • 100% On-Device (or Bring Your Own Homelab/Server STT)

VoxCtrl acts as an intelligent desktop voice gateway, routing speech to any destination—typing directly into focused windows, invoking terminal agents, appending to journals, triggering shell commands, streaming to webhooks, or feeding local AI assistants.

---

🌟 Key Features

• High-Performance Offline Speech Recognition:
  • whisper.cpp: Native local inference via whisper-rs (GGUF models) with Vulkan/CUDA GPU compute and CPU fallback.
  • Moonshine: Streaming ONNX speech recognition. Runs on the CPU by default; optional WebGPU acceleration (Direct3D 12 on Windows, the Linux WebGPU AppImage on newer distros).
  • Parakeet TDT: Ultra-fast non-autoregressive transcription via NVIDIA Parakeet ONNX models (including the 1.58-bit Redux variant). CPU by default; optional WebGPU acceleration.
  • One GPU switch per engine, and a Speed test: every engine has the same GPU acceleration checkbox and a chip showing the device in use. Settings → Engine → Speed test times each engine on the CPU and GPU on your machine and recommends the faster one — the GPU is several times faster for some engines and slower for others.
  • Remote Speech Engine: Offload transcription to any OpenAI-compatible /v1/audio/transcriptions network endpoint (Faster-Whisper, vLLM, Whisper standalone, or cloud APIs) with zero local RAM/VRAM overhead.
• On-Device S1-mini Dictation Cleanup:
  • Intelligent text normalization powered by Superwhisper's s1-mini (~480 MB download).
  • Runs in an isolated voxctrl-llm-sidecar process using llama.cpp with Vulkan GPU offload (switchable to CPU in Settings) and automatic CPU fallback.
  • Cleans spoken self-corrections, fixes punctuation and casing, and strips filler words while strictly preserving command keywords.
• Private Dictation History: The History window (tray menu → History) lists recent dictations to copy, type again, read aloud or send to another target. It's kept in memory only and forgotten after 24 hours or 50 entries by default. Keeping it between restarts is opt-in: the file is encrypted and its key lives in your desktop keyring. Any target can opt out with Don't keep in history. See Privacy.
• Programmable Output Command Router (11 Delivery Targets):
  • Route dictation to focused windows (inject), system clipboard (clipboard), shell commands (exec), FIFO pipes (pipe), TCP/Unix sockets (socket), markdown files (file), desktop bus (dbus), HTTP APIs (http), HMAC-signed webhooks (webhook), audio playback (speak), or conversational LLMs (chat).
  • Spoken Voice Commands: Say "VoxCtrl notes, meeting recap" to dynamically dispatch text to the command named notes.
  • Multi-Target Broadcasting: Bind a single hotkey gesture to broadcast a single dictation sequentially to multiple output targets.
• Neural Text-to-Speech (TTS) Suite:
  • 6 offline voice engines: Breeze-TTS-2 (voice cloning), VoxCPM2 (OpenBMB voice cloning), Pocket-TTS (voice cloning from reference clip), Piper (high-quality ONNX), Inflect-Micro-v2 (ultra-lightweight 38 MB ONNX), and eSpeak-NG (instant fallback).
  • On-Demand Memory Mode: Automatically unloads heavy TTS neural models from RAM/VRAM after a configurable idle period (tts.idle_unload_secs).
• Heads-Up HUD Overlay & Visuals:
  • Transparent, click-through, voice-reactive animated HUD overlays with 8 distinct styles: Ocean Wave (tide pool with bobbing buoy), Voice Card (holographic card with 20×6 VU meter), Waveform (oscilloscope CRT trace), Pulse Ring (sonar radar dial), Mono Bars, Neon Spectrum, Retro Terminal and Analog VU.
  • Drop-in custom overlays: an index.html + style.css pair in the overlays folder, re-read from disk on every activation, so an edit shows up on the next dictation with no restart.
  • Floating command trigger HUD pills (⚡ TARGET ▸ Text) showing dispatched actions.
• Privacy-Preserving Global Hotkeys:
  • Registered via the XDG GlobalShortcuts portal on Linux (Wayland & X11) and native hooks on Windows.
  • VoxCtrl never monitors your keyboard keystrokes—the desktop simply notifies VoxCtrl when its registered shortcut is triggered.
  • Supports hold, toggle, double_tap, and double_tap_hold gestures.
• Built-in Model Context Protocol (MCP) Server:
  • Local JSON-RPC server (a Unix socket in $XDG_RUNTIME_DIR on Linux, also at /tmp/voxctrl-mcp.sock, or a named pipe on Windows) exposing transcribe_voice, speak_text, get_status and set_target tools to AI clients like Claude Desktop and Cursor, plus get_recent_transcripts if you allow it.
• First-Run Setup Wizard & In-App Updates:
  • 7-step interactive wizard (voxctrl --setup) for instant microphone, engine, hotkey, and overlay configuration.
  • Self-contained updater that downloads, cryptographically verifies (SHA-256), and atomically replaces application binaries on release.
  • Redacted, allowlist-filtered one-click bug reporting.

---

🎯 Output Routing Targets

VoxCtrl turns your voice into a programmable router via targets.toml:

Delivery Type	Mechanism	Primary Use Case
inject	Pastes the text (one Ctrl+V; your whole clipboard is backed up and restored), falling back to typing it via wtype/ydotool (Wayland), xdotool (X11) or SendInput (Windows). See Pasting Dictation.	Standard dictation directly into any active editor, browser, or terminal.
clipboard	System clipboard population via arboard.	Quiet copying of notes, code snippets, or templates without modifying cursor focus.
exec	Spawns shell command with {TEXT} substitution (shell=False safety).	CLI automation (git commit -m "{TEXT}", piping into local tools, web searches).
pipe	Writes transcription bytes to a local named FIFO pipe.	Interfacing with shell scripts, terminal agents, and background listeners.
socket	Streams text over TCP or Unix Domain Sockets.	Daemons, remote servers, containers, or background dev environments.
file	Appends transcriptions to files with prefixes and UTC timestamps.	Hands-free journaling, meeting logs, daily standup notes, or task lists.
dbus	Emits custom ai.voxctrl.Dictation DBus signals on the session bus.	Desktop notification triggers, system scripts, and desktop widget updates.
http	Dispatches HTTP POST/GET requests formatted with JSON payloads.	Webhooks, database ingestion pipelines, and REST service integration.

---

🛠️ Architecture

VoxCtrl is designed with strict modularity, memory isolation, and high concurrency across 15 specialized Rust workspace crates:

Workspace Crates

Crate	Responsibility
voxctrl-app	Tauri 2 application shell, Svelte IPC commands, system tray, and window management.
voxctrl-core	Shared domain types, audio buffer representations, and engine traits.
voxctrl-audio	cpal audio input stream, ring buffers, device enumeration, VAD, and RNNoise.
voxctrl-inference	Multi-engine STT runner (whisper.cpp, Moonshine, Parakeet TDT, and remote HTTP) and the CPU-vs-GPU Speed test (bench.rs).
voxctrl-llm-sidecar	Independent companion process running llama.cpp (llama_cpp_2) with Vulkan GPU offload and CPU fallback for S1-mini text cleanup.
voxctrl-llm	IPC client communicating with voxctrl-llm-sidecar and external OpenAI-compatible LLM endpoints.
voxctrl-routing	11-way delivery router, voice command prefix matcher, and multi-target dispatch.
voxctrl-hotkeys	XDG Desktop Portal GlobalShortcuts integration and gesture state machine.
voxctrl-tts	Neural TTS orchestration (Breeze-TTS-2, VoxCPM2, Pocket-TTS, Piper, Inflect, eSpeak) and idle memory unloading.
voxctrl-inject	Text delivery: paste (with clipboard restore) or typing; shortcut choice per focused window; key sending via XTEST, wtype, ydotool, the RemoteDesktop portal, xdotool.
voxctrl-clipboard	Full-format clipboard backup/restore on Windows, X11 and Wayland.

---

📥 Installation

Pre-built binaries are available on the Latest Releases page.

Linux

1. Download VoxCtrl-linux-x86_64-vulkan.AppImage or the .deb package from Releases.

2. Make it executable and run:

   chmod +x VoxCtrl-linux-x86_64-vulkan.AppImage
   ./VoxCtrl-linux-x86_64-vulkan.AppImage
   

   The AppImage automatically utilizes Vulkan GPU acceleration if available, falling back gracefully to CPU compute.

   On a newer distro (glibc 2.38+: Ubuntu 24.04, Fedora 39, Debian 13, Arch and similar) you can instead download VoxCtrl-linux-x86_64-vulkan-webgpu.AppImage, which also lets Moonshine and Parakeet use the GPU. Use the plain Vulkan AppImage on Ubuntu 22.04 and Debian 12.

3. On first launch, the First-Run Setup Wizard guides you through microphone selection, engine configuration, and hotkey binding. You can also re-launch it anytime with:

   voxctrl --setup
   

Windows

1. Download VoxCtrl-windows-x86_64-webgpu.exe from Releases. It can use Direct3D 12 WebGPU acceleration for Moonshine and Parakeet if a usable GPU is available (switch it on per engine in Settings → Engine, or run the Speed test), and otherwise runs on the CPU.
2. Run the installer and launch VoxCtrl from the Start Menu or System Tray.

---

🔨 Building from Source

Prerequisites

• Rust: rustup default stable (1.80+)
• Node.js & npm: Node.js 18+ and npm
• CMake: Required for building whisper-rs and llama.cpp

Linux Dependencies

Install required system packages:

• Ubuntu / Debian:

  sudo apt install -y build-essential cmake pkg-config libasound2-dev libvulkan-dev shaderc \
                      libssl-dev libglib2.0-dev libwebkit2gtk-4.1-dev libgtk-3-dev squashfs-tools
  # Runtime injection helpers (typing fallback; install at least one).
  # Pasting itself needs none of them on X11; on Wayland it uses wtype, ydotool
  # or the desktop's RemoteDesktop portal (permission dialog on first use).
  sudo apt install -y wtype    # For Wayland (wlroots compositors)
  sudo apt install -y xdotool  # For X11
  

• Arch / CachyOS:

  sudo pacman -S --needed base-devel cmake pkg-config alsa-lib vulkan-headers shaderc \
                          openssl glib2 webkit2gtk-4.1 gtk3 squashfs-tools wtype xdotool
  

• Fedora:

  sudo dnf install -y gcc-c++ cmake pkgconfig alsa-lib-devel vulkan-headers shaderc \
                      openssl-devel glib2-devel webkit2gtk4.1-devel gtk3-devel squashfs-tools wtype xdotool
  

Windows Dependencies

• Visual Studio 2022 C++ Build Tools (MSVC cl.exe)
• CMake added to system PATH
• Vulkan SDK (recommended for S1-mini GPU acceleration)

---

Building on Linux

1. Development Mode (with Live Reload)

git clone https://github.com/JRufer/VoxCtrl.git
cd VoxCtrl
npm install
cargo tauri dev

2. Standalone Portable AppImage (Recommended)

Compile the standalone, hardware-accelerated, self-contained AppImage:

./build_appimage.sh

This compiles voxctrl-llm-sidecar and voxctrl, packages frontend assets, bundles required GTK/WebKit helpers, applies host-fallback library stripping, and outputs VoxCtrl-linux-x86_64-vulkan.AppImage in the root folder.

3. Standard Production Build (deb & AppImage)

npm run build
# The sidecar is rebuilt by tauri's beforeBuildCommand; this keeps it on Vulkan.
VOXCTRL_SIDECAR_FEATURES=vulkan npx tauri build --features vulkan

---

Building on Windows

Open an x64 Native Tools Command Prompt for VS 2022:

1. Standard CPU Build

git clone https://github.com/JRufer/VoxCtrl.git
cd VoxCtrl
npm install
npm run build
cargo build --bin voxctrl-llm-sidecar --release
npx tauri build --bundles nsis

2. GPU-Accelerated Build (WebGPU Direct3D 12)

Enables Direct3D 12 GPU acceleration for Moonshine and Parakeet via WebGPU and Vulkan for S1-mini:

npm run build
# The sidecar is rebuilt by tauri's beforeBuildCommand; this keeps it on Vulkan.
set VOXCTRL_SIDECAR_FEATURES=vulkan
npx tauri build --bundles nsis --features moonshine-webgpu,parakeet-webgpu

The resulting installer is saved to src-tauri/target/release/bundle/nsis/.

---

⚙️ Configuration

Configuration files are located in ~/.config/voxctrl/ (Linux) or %APPDATA%\voxctrl\ (Windows). Changes hot-reload immediately.

config.json (Core Application Settings)

{
  "engine": {
    "backend": "moonshine",
    "moonshine": { "model_size": "base", "language": "en" },
    "whisper_cpp": { "model_size": "base", "device": "auto" },
    "parakeet": { "model_size": "tdt-0.6b-v3", "language": "auto" },
    "remote_openai": {
      "endpoint": "http://192.168.1.50:8000/v1",
      "model": "whisper-1",
      "timeout_secs": 30
    },
    "s1_mini": { "enabled": true, "styling": "semi-formal" }
  },
  "tts": {
    "engine": "pocket_tts",
    "memory_mode": "on_demand",
    "idle_unload_secs": 60
  },
  "ui": {
    "overlay_style": "ocean_wave",
    "command_overlay_duration_secs": 3.0
  }
}

targets.toml (Output Routing Commands)

format_version = "1.1"

[[target]]
id = "default"
label = "Active Window"
delivery = "inject"

[[target]]
id = "notes"
label = "Meeting Notes"
delivery = "file"
file_path = "~/Documents/meeting_notes.md"
file_prefix = "- "
file_timestamp = true

[[target]]
id = "cmd_router"
label = "Voice Command Router"
delivery = "command"    # Listens for "VoxCtrl <target> <text>"

bindings.toml (Global Shortcut Gestures)

format_version = "1.1"

[[binding]]
id = "dictate_hold"
label = "Dictate (Hold Space)"
keys = ["KEY_LEFTMETA", "KEY_SPACE"]
gesture = "hold"
target_id = "default"

[[binding]]
id = "type_and_log"
label = "Type & Append to Journal"
keys = ["KEY_LEFTCTRL", "KEY_LEFTMETA", "KEY_SPACE"]
gesture = "hold"
target_ids = ["default", "notes"]    # Broadcasts to both targets sequentially!

---

📚 Documentation Index

For in-depth guides, architectural references, and developer documentation:

Guide	Description
Architecture	Workspace crate design, concurrency model, and data flow.
Speech Recognition	Whisper.cpp, Moonshine, Parakeet, Remote STT, and S1-mini sidecar.
Output Routing	Comprehensive reference for all 11 delivery mechanisms.
Text-to-Speech	Engine setup, voice cloning, prompt design, and on-demand memory.
Global Hotkeys	XDG portal shortcuts, gesture recognizer, and platform details.
Integrations	Model Context Protocol (MCP), DBus, and OpenAI LLM API integration.
User Interface	HUD overlays, Cyber Obsidian settings dashboard, and system tray.
Overlays	Every built-in overlay style, the window's lifecycle, and writing your own.
Configuration Reference

---

📄 License

VoxCtrl is open-source software licensed under the MIT License.

---

Buy me a coffee

If you find VoxCtrl useful, please consider supporting its development with a small contribution:

Thank you for your support! ❤️