xa11y

A Playwright-style library for driving native desktop apps on macOS, Windows, and Linux. Built for end-to-end tests, computer-use agents, and assistive tools.

Use cases: end-to-end desktop testing, computer-use agents, MCP tools, assistive technology.

Documentation | Rust API | Python API | JavaScript API

Quick Example

import xa11y

safari = xa11y.App.by_name("Safari")

# Find elements with CSS-like selectors via locator
for button in safari.locator("button").elements():
    print(button.name)

# Interact with elements via locator (re-resolves every call)
safari.locator("button[name='Submit']").press()

safari.locator("text_field[name^='Search']").set_value("hello world")

Installation

Rust

cargo add xa11y

Python

pip install xa11y

Requires Python 3.9+. Pre-built wheels available for Linux, macOS, and Windows.

For pytest suites, pytest-xa11y adds fixtures that launch the app under test, capability markers that skip what the machine cannot do, and the accessibility tree on every failure:

pip install pytest-xa11y

JavaScript

npm install @crowecawcaw/xa11y

Requires Node.js 18+. Pre-built native binaries available for Linux, macOS, and Windows.

On macOS, grant your terminal two permissions in System Settings > Privacy & Security:

1. Accessibility, required for all accessibility API access. On macOS 27 and newer, grant it in Device Control and Data Access.
2. Screen & System Audio Recording (macOS 26+), required to read window content. Without it, only menu bars are visible.

Restart your terminal after changing permissions.

On Linux, AT-SPI2 must be running (the default on GNOME and most desktop environments). Nothing else to grant.

Windows works as installed.

Selector Syntax

Query accessibility trees with CSS-like selectors:

Pattern	Meaning
button	Elements with role Button
button[name='OK']	Button named exactly "OK"
text_field[name^='Search']	Text field whose name starts with "Search"
text_field[name*='email']	Text field whose name contains "email"
group > button	Buttons that are direct children of a group
window button	Buttons anywhere inside a window
button:nth(2)	The 2nd button match

Supported Actions

Action	Description
press	Click / invoke
focus / blur	Move or remove keyboard focus
toggle	Toggle a checkbox or switch
expand / collapse	Expand or collapse a disclosure
select	Select an item
set_value	Set a text field's value
type_text	Type text into an element
increment / decrement	Adjust a slider or stepper
show_menu	Open a context menu

Platform Support

Platform	Backend
macOS	AXUIElement
Linux	AT-SPI2 (D-Bus)
Windows	UI Automation

Ecosystem

• strands-xa11y packages xa11y as a desktop-control tool for Strands, AWS's open-source SDK for building AI agents. One tool gives an agent the accessibility tree and the actions on it; a read-only variant covers agents that observe and never act. pip install strands-xa11y (requires Python 3.10+).
• agent-desktop is a CLI built on xa11y for AI agents that read and control desktops.

Contributing

git clone https://github.com/xa11y/xa11y && cd xa11y
cargo build --workspace
cargo xtask check   # fmt, lint, test, python bindings

See the development docs for architecture and testing strategy.

License

MIT. All dependencies are permissively licensed (MIT, Apache-2.0, BSD, or similar), enforced via cargo-deny.