xa11y
A Playwright-style library for driving native desktop apps on macOS, Windows, and Linux. Built for end-to-end tests, computer-use agents, and assistive tools.
Use cases: end-to-end desktop testing, computer-use agents, MCP tools, assistive technology.
Documentation | Rust API | Python API | JavaScript API
Quick Example
import xa11y
safari = xa11y.App.by_name("Safari")
# Find elements with CSS-like selectors via locator
for button in safari.locator("button").elements():
print(button.name)
# Interact with elements via locator (re-resolves every call)
safari.locator("button[name='Submit']").press()
safari.locator("text_field[name^='Search']").set_value("hello world")
Installation
Rust
cargo add xa11y
Python
pip install xa11y
Requires Python 3.9+. Pre-built wheels available for Linux, macOS, and Windows.
For pytest suites, pytest-xa11y adds
fixtures that launch the app under test, capability markers that skip what the
machine cannot do, and the accessibility tree on every failure:
pip install pytest-xa11y
JavaScript
npm install @crowecawcaw/xa11y
Requires Node.js 18+. Pre-built native binaries available for Linux, macOS, and Windows.
On macOS, grant your terminal two permissions in System Settings > Privacy & Security:
- Accessibility, required for all accessibility API access. On macOS 27 and newer, grant it in Device Control and Data Access.
- Screen & System Audio Recording (macOS 26+), required to read window content. Without it, only menu bars are visible.
Restart your terminal after changing permissions.
On Linux, AT-SPI2 must be running (the default on GNOME and most desktop environments). Nothing else to grant.
Windows works as installed.
Selector Syntax
Query accessibility trees with CSS-like selectors:
| Pattern | Meaning |
|---|
button | Elements with role Button |
button[name='OK'] | Button named exactly "OK" |
text_field[name^='Search'] | Text field whose name starts with "Search" |
text_field[name*='email'] | Text field whose name contains "email" |
group > button | Buttons that are direct children of a group |
window button | Buttons anywhere inside a window |
button:nth(2) | The 2nd button match |
Supported Actions
| Action | Description |
|---|
press | Click / invoke |
focus / blur | Move or remove keyboard focus |
toggle | Toggle a checkbox or switch |
expand / collapse | Expand or collapse a disclosure |
select | Select an item |
set_value | Set a text field's value |
type_text | Type text into an element |
increment / decrement | Adjust a slider or stepper |
show_menu | Open a context menu |
Platform Support
| Platform | Backend |
|---|
| macOS | AXUIElement |
| Linux | AT-SPI2 (D-Bus) |
| Windows | UI Automation |
Ecosystem
- strands-xa11y packages xa11y as a desktop-control tool for Strands, AWS's open-source SDK for building AI agents. One tool gives an agent the accessibility tree and the actions on it; a read-only variant covers agents that observe and never act.
pip install strands-xa11y (requires Python 3.10+).
- agent-desktop is a CLI built on xa11y for AI agents that read and control desktops.
Contributing
git clone https://github.com/xa11y/xa11y && cd xa11y
cargo build --workspace
cargo xtask check # fmt, lint, test, python bindings
See the development docs for architecture and testing strategy.
License
MIT. All dependencies are permissively licensed (MIT, Apache-2.0, BSD, or similar), enforced via cargo-deny.
