Tuxistant

A voice assistant for the Linux desktop. Press a shortcut or hold Fn, talk, and Tux answers out loud while an animated blob reacts on screen and captions show what was heard and said.

Speech recognition, the language model and speech synthesis run on a LocalAI server of your own, which server/ sets up. Tuxistant is the desktop side: a GNOME Shell extension, a small daemon and a Settings app.

Requirements

• GNOME Shell 50 on Wayland
• Python 3.14 with dbus_fast, websockets, PyYAML and PyGObject (GTK 4, libadwaita 1.9 or newer). Ubuntu 26.04 has all of this except two modules, which the first install step adds
• PipeWire with pw-record and pw-cat
• glib-compile-schemas
• A LocalAI server with a realtime pipeline (tux-realtime by default), a voice model (tux-voice) and a speech-to-text model (whisper-large-turbo-q5_0), which dictation and, from 0.3.0, every Fn hold use. server/ has a script that sets one up on a computer with an 8 GB NVIDIA GPU
• For the skills, all optional: bwrap (bubblewrap) for shell commands that need no click, localsearch or plocate for finding files, GSConnect for phones, python3-cryptography for Tapo lights, Evolution Data Server with its Python bindings (gir1.2-ecal-2.0 on Ubuntu) for the calendar, Thunderbird for mail search, wpctl, which comes with WirePlumber, for the volume, a Home Assistant of your own for the smart home, and a LibreTranslate server for translation, with Piper voices on LocalAI to hear it spoken (server/ sets up the translation server and the voices)
• For Tapo lights and your own skills, from 0.3.0: the skill sandbox, which needs bwrap and prlimit on an x86_64 or aarch64 computer, and Linux 6.12 or newer with Landlock on for lights and any skill that uses the network. 0.2.0 ran the lights without it, so without these they stop working after the update
• For web search, research and the web fallback for facts, from 0.3.0: a SearXNG server of your own, which server/ doesn't set up. Tuxistant looks for it on port 8888 of the voice server unless Settings names another, and it needs JSON output turned on (formats: [html, json] under search: in its settings.yml)

Install

There are three releases to choose from:

• 0.2.0, the stable release. What it added to 0.1.0 has all worked on the owner's own desktop, and it is what the website offers.
• 0.3.0, a preview. It adds media, timers, Home Assistant, mail search, facts from Wikipedia, web search and research, and your own skills. Most of it has only run in tests so far; CHANGELOG.md says which parts.
• 0.4.0, a preview. It adds opening apps and Settings pages, and a click before the facts lookup sends a question that may be about your home. None of it has been tried on a real desktop yet.

On Ubuntu 26.04, add the two Python modules it doesn't come with, and git to fetch the code. This takes 0.2.0; for a preview, use --branch v0.3.0 or --branch v0.4.0 instead:

sudo apt install git python3-dbus-fast python3-websockets
git clone --branch v0.2.0 https://github.com/BuenGenio/Tuxistant.git
cd Tuxistant

Or take a download from the releases page, or 0.2.0 from the website, and skip git: tar xzf Tuxistant-0.2.0.tar.gz && cd Tuxistant-0.2.0, or tar xzf Tuxistant-0.4.0.tar.gz && cd Tuxistant-0.4.0 for the newest preview.

Other systems name these packages differently. The installer stops before copying anything if a module is missing, and says which.

./install.sh                  # everything, including hold-Fn-to-talk; the Fn helper uses sudo
./install.sh --no-fn-helper   # the same without hold-Fn-to-talk, so no sudo

The first form asks whether to add the Claude SDK for Cloud, which is optional, so answer no unless you want it. Then it asks for your password to install the Fn helper.

Then finish by hand, because a running Wayland session cannot load a new extension:

1. Log out and back in once.
2. Run gnome-extensions enable tuxistant@buengenio.
3. Run tuxistant doctor to check that everything is in place, including that the extension is enabled.

On a new account ~/.local/bin isn't on your PATH until you log back in. Until then the command is ~/.local/bin/tuxistant.

The installer puts a tuxistant command in ~/.local/bin, the extension in ~/.local/share/gnome-shell/extensions, a systemd user unit and a D-Bus activation file in ~/.local/share, and creates ~/.config/tuxistant/config.json. It sets the daemon to start when you log in. Set the server and API key in Settings under General. The server starts as localhost:8080, which is right when LocalAI runs on the same computer. To have the installer fetch the key, set TUXISTANT_KEY_COMMAND to a command that prints it.

The tuxistant command runs from this folder, so keep it where it is. To update a clone, run git fetch --tags, git checkout the newer tag, and run ./install.sh again. To update from a download, unpack the newer one, which makes a new folder beside the old one, run ./install.sh from the new folder, restart the daemon with systemctl --user restart tuxistant (or log out and back in), and only then delete the old folder, which the old daemon runs from until it restarts. Don't unpack it over the old folder, because files the new version removed would stay behind. After updating a clone, restart the daemon the same way. Until then the old version keeps running. Running the installer twice is harmless and keeps your settings.

To remove everything, including the Fn helper, run ./uninstall.sh. Your settings, API key, history, themes and what skills kept stay unless you add --purge.

This is the beginning of the README. Read the rest at the original. Read the full README ↗