Aloe Desktop
Aloe Desktop is the native desktop agent for Aloe. It is Aloe with access to your computer: the same chat as the web app, plus the ability to safely work with granted folders, run local commands, manage terminal sessions, open local URLs, show notifications, and report recent activity back to the user.
The app is built with Tauri 2, Rust, React 19, Vite, and Bun.
Features
- Sign in with Google (through the browser), the same account as the web app.
- Only chats sent from this app can use the computer. Chats on the Aloe web app never get local tools; the app proves where a chat came from by sending its device credential in
X-Aloe-Agent.
- Maintain a websocket connection to the Aloe backend.
- Grant and revoke local folder access, permanently for the device or temporarily for a single chat.
- Search, read, create, update, delete, and patch files inside granted folders.
- Run commands and terminal sessions with approval controls.
- Drive a real local Chrome over the DevTools Protocol so Aloe can use sites behind the user's own logins.
- Choose command trust mode: ask every time, trusted coding commands, or allow all.
- Show pending approvals and recent agent activity in the desktop UI.
- Support tray behavior, startup preferences, notifications, and silent auto-updates.
Project Structure
.
|-- src/ # React desktop UI
|-- src-tauri/ # Tauri/Rust application shell and local agent runtime
|-- build_release.py # Interactive release build helper
|-- UPDATER.md # Auto-update release process
|-- package.json # Bun scripts and frontend dependencies
`-- vite.config.ts # Vite configuration
Requirements
- Bun
- Rust stable and Cargo
- Tauri 2 system dependencies for your OS
- An Aloe account (Google sign-in)
For Linux, install the native packages required by Tauri/WebKitGTK for your distribution before running or building the app.
Development
Install dependencies:
bun install
Fetch Cua Driver, which desktop control uses and the app bundles (the build fails without it):
python scripts/fetch_cua_driver.py
Run the desktop app in development mode:
bun run tauri:dev
The Tauri dev server starts Vite on 127.0.0.1:1420 through the beforeDevCommand configured in src-tauri/tauri.conf.json.
Run a frontend typecheck and production Vite build:
bun run build
Build the native app:
bun run tauri:build
Connecting to Aloe
- Open Aloe Desktop.
- Click Continue with Google. Your browser finishes sign-in and hands the session back over the
aloe:// deep link.
- Aloe opens, signed in, and connects.
- Grant folders that Aloe is allowed to inspect or modify.
- Review command approvals from the Desktop controls page when commands are requested.
In debug builds, the app defaults to http://127.0.0.1:8080 for the Aloe backend. In release builds, it defaults to https://api.247autoarmy.in/.
To compile against a different backend URL, set ALOE_BACKEND_URL at build time:
ALOE_BACKEND_URL=https://your-api.example.com bun run tauri:build
Local Data
Aloe Desktop stores its local configuration in the operating system config directory under Aloe Desktop/config.json. The config includes the registered agent id, credential, user profile, granted folders, desktop preferences, recent actions, and terminal session metadata.
Use Log out in the app to reset the agent connection and remove stored credentials.
Command Approval Modes
-
ask: every command request is queued for explicit approval. A denial tells Aloe it was denied, so it stops rather than retrying the same command through another tool.
-
auto: Aloe's own model judges each command against the conversation it came from (via the backend) before running it. This judgement is the only gate — there is no local blocklist in front of it, because the previous one rejected pipes, &&, and anything containing "curl" or "deploy", which is most real development work and left auto mode behaving exactly like ask. A command the check clears runs unattended. A command it declines does not reach the approval queue — it is refused on the spot and the judge's reason is returned to Aloe as the tool's error, so the model can explain what was blocked and propose something narrower in the same turn. Note the consequence: in auto there is no way to approve a command the judge rejected; switch to ask if you want the final say on every command.
The two not-safe outcomes are kept distinct. A verdict refuses. A failed check — network error, timeout, unparseable answer — has judged nothing, so it falls back to the approval queue rather than refusing a command on no evidence.
-
all: command approvals are disabled.
File operations are still limited to folders the user has explicitly granted.
Folder Access
Two kinds of grant, with the same enforcement:
- Device folders, added in this app, reachable by every conversation and every background task for as long as they are listed.
- Chat folders, shared from the chat composer in this app. Pressing the folder button there asks this device to open its native folder dialog; the folder the user picks becomes reachable from that one conversation and nowhere else, until they remove it in the same place. They are listed read-only under Folder access so it is always visible on the device what a chat can reach.
The browser never names a path — it can only ask for the dialog, and is told the result. Aloe Desktop keeps its own copy of the chat grants and re-checks every job against them, so a job naming a conversation it was not granted still fails here.
Sensitive-path write rules (.git/, node_modules/, build output, .env, keys) apply identically to both kinds.
Aloe Code
The Chat | Code switch in the title bar opens Aloe Code: Aloe as a coding agent working in a project folder on this computer, like Claude Code in the Claude desktop app.
- A project is a granted device folder (
Add project folder uses the same grant as Folder access and shows up there). Sessions are listed in Code mode's own sidebar, grouped by project, and kept out of the chat sidebar.
- A session is a conversation marked
workspace: "code", but it is not a chat mode. Requests carry workspace: "code" and the session's projectPath, and the backend answers them with its own fixed setup (lib/chat/code_workspace.ts): the Deep model chain, the full local toolset plus web, and a coding system prompt holding the project path and the project's own MEMORY.md/AGENTS.md. Chat's per-turn context (memories, personality, board, connectors, skills, clock) is left out, and so is tool discovery.
- The composer has no mode slider, no per-chat folder sharing and no @mentions. Attaching files and screenshots still works. Approvals and trust modes are the same as in chat.
- Only the git branch rides along with each message as hidden context (an
<aloe-context> envelope the thread doesn't display), since it can change mid-session.
- Built for prompt caching: the tools and system prompt are identical on every turn, every turn since the last compaction is replayed in full (file reads included) instead of aging into a digest, and Anthropic models start on the 1-hour cache TTL, so a session re-reads its history from cache rather than at full price.
- Where it's stored: messages live on the Aloe backend like any chat. The backend also marks the conversation as a Code session (
workspace: "code"), which keeps it out of the chat list everywhere, including the web app. The project folder path for each session stays on this device, in localStorage. A session marked on the server that this device has no project for (from another computer, or after a reinstall) is listed under "No project on this computer", and you choose its project before continuing.
- The branch is read from
.git/HEAD rather than by running git, so opening a project never runs code from its repository config.
Desktop Control
When the user switches on "Let Aloe control this computer", the backend's desktop agent drives apps through Cua Driver, bundled under resources/cua-driver/ and pinned in scripts/fetch_cua_driver.py. Aloe runs one cua-driver mcp process (src-tauri/src/cua.rs) and forwards desktop_cua jobs to it: per-window accessibility trees, element-token clicks and typing, delivered in the background without taking the user's pointer or focus. Cua's telemetry is turned off (DO_NOT_TRACK=1).
Aloe filters every call before it reaches the driver: only an allowlist of tools, never Aloe's own process or windows, no file writes, and launch_app by app name only (never an executable path or arguments). Moving the pointer into a screen corner still stops control. Set ALOE_CUA_DRIVER to a driver binary to develop against a different build.
Voice (push to talk)
Hold Left Ctrl + Left Alt (Left Control + Left Option on macOS) anywhere, talk, and let go. A glow around the screen and a panel at the bottom show your words as they are heard; Aloe's answer streams into the same panel and is read aloud. It is on by default and can be switched off under Settings › Desktop app › Voice.
- The chord. It must be held for 200 ms with nothing else pressed, so ordinary Ctrl+Alt shortcuts never trigger it. Pressing another key mid-hold discards the recording. AltGr (Left Ctrl + Right Alt) never matches, and Aloe's own synthetic typing is ignored. A quick tap stops Aloe talking; holding again interrupts the reply and stops that turn on the backend.
- Where the words go. Each turn joins one standing conversation (
voiceConversationId in config.json), sent as a desktop chat with X-Aloe-Agent, so it has the same local tools, folder grants and command approvals as a chat typed in this app.
- Speech to text. The microphone is captured in Rust (
src-tauri/src/voice/audio.rs) and streamed straight to OpenAI's realtime transcriber (gpt-live-transcribe), authenticated with a one-minute ephemeral key from POST /api/voice/live-session. The OpenAI key itself lives only on the backend (OPENAI_API_KEY); the backend never receives or stores the audio.
- Text to speech. Each finished sentence of the reply is fetched from
POST /api/voice/speak (OpenAI TTS, 24 kHz PCM) and played in order.
- The overlay (
voice.html, src-tauri/src/voice/overlay.rs) never takes keyboard focus and is excluded from Aloe's own screenshots, like the desktop-control overlay.
- Platforms. Windows uses a low-level keyboard hook, which only observes; when the chord is released it taps an unassigned key so the app in front doesn't open its menu bar. macOS uses a listen-only event tap, which needs Input Monitoring permission (System Settings › Privacy & Security); the Settings pane says when it is missing. Linux isn't supported yet.
Browser Automation
Aloe drives Chrome (or Chromium/Edge) through the DevTools Protocol on 127.0.0.1:9333. If something is already listening there, Aloe attaches to it; otherwise it launches the browser itself.
Two profiles are available:
default — the user's everyday Chrome profile, with their existing logins. Chrome must not already be running with that profile, because a second instance cannot open a debugging port on a locked profile. Aloe reports this explicitly if the port never comes up.aloe — a separate profile under Aloe Desktop/browser-profile, which can run alongside a normal Chrome window but starts signed out. Logins persist there between sessions.
Browser actions are not gated by the command-approval modes above: navigation, clicks, and typing run as soon as they are requested. Page content is untrusted input — a page that instructs the assistant to do something is data about that page, not a request from the user.
Aloe never launches the browser headless. The window is visible so the user can watch and take over.
Releases and Updates
Auto-update behavior is configured in src-tauri/tauri.conf.json. Release builds create updater artifacts, and the app checks:
https://github.com/Trifles-Labs/aloe-desktop/releases/latest/download/latest.json
See UPDATER.md for signing key setup, versioning, tagging, publishing, and local update testing.
You can also use the release helper:
python build_release.py 1.2.3
On Linux, choose bundles with:
python build_release.py 1.2.3 --linux-bundles deb,rpm
Troubleshooting
If sign-in fails, verify the backend URL, the web app URL (ALOE_FRONTEND_URL), and that the backend is reachable from the desktop machine.
If the socket stays disconnected, log out and sign in again. The app also reconnects automatically with backoff when the backend is temporarily unavailable.
If folder operations fail, remove and re-add the folder so the stored path is refreshed and canonicalized.
If Linux rendering fails under Wayland, the app enables software rendering for Wayland sessions at startup.
Useful Commands
bun install # install JavaScript dependencies
bun run tauri:dev # run the desktop app locally
bun run build # typecheck and build the web UI
bun run tauri:build # build native installers/artifacts
python build_release.py # guided release build
