QAIA — Agentic QA platform, open source

🇫🇷 Lire ce README en français

A user story goes in. A traceable Gherkin test book and runnable Playwright tests come out — as Claude Code plugins that run inside your session. No API key, no backend, no data leaving your session beyond what you already send to Claude.

▶ Try it live — paste a Gherkin test book and watch QAIA judge it. A high score can still fail; the demo shows why.

The whole product in one screen

One acceptance criterion from eval/gold-set/US-004-expense-approval.md:

A report under €500 total needs one approval (the employee's direct manager). €500–€5000 needs manager then finance.

Under €500 and €500–€5000 do not say what happens at exactly €500.00. Here is what came out (approval-chain.feature, verbatim):

  @QAIA-US-004-009 @AC2 @P1 @boundary @low-confidence
  # condition: AC2-C2 — priority P1 — open: Q1 (exact-€500 boundary — read as inclusive
  # in band B: manager then finance)
  Scenario: A report of exactly €500.00 needs manager then finance
    Given a submitted report "R" by "employee@demo" totalling exactly 500.00 EUR
    When "manager@demo" approves report "R"
    Then report "R" still awaits approval from "finance"

The tool did not silently pick a reading of the boundary. It picked one, tagged it @low-confidence, numbered the open question, and wrote its assumption in the file — so a human can overturn it in one line instead of finding it in production. A stable ID that survives regeneration, the criterion it came from, the technique that produced it, and the ambiguity declared rather than guessed away.

38 scenarios came from that one story, 11 flagged low-confidence. Disclosed, because a demo should be: that ticket is our own gold-set fixture with ambiguities planted on purpose, the run was non-interactive (every human decision recorded as simulated), and the acting model had read the file's sequestered judge section — all three stated in the run's own journey file. It shows the shape of the output, not that it works on your ticket.

Two real defects, in software we did not write

Everything above is measured on code this project produced itself, which is the weakest kind of evidence. So the pipeline was pointed at typicode/json-server — 75,694 stars — and allowed to read only its README. Never the code, never the issues, never the fix commits.

• A one-character contract break. The docs promised _dependent; the code read dependent. The endpoint answered success, deleted the post, and left every dependent record in place. A suite written by reading the code cannot find this — it copies the mistake. Filed by a real user as issue #1551; fixed in 1b7c0fb.
• Two filters that overwrote each other. views_gt=100&views_lt=300 returned everything. Fixed in e6055e6.
• A third finding, refused. _start alone returns an empty list — a fact — but the README only ever shows it paired. Counted as contested. Two, not three.

And the part that goes against us. Against the current version four tests fail and three are our fault: those features left the documentation and the suite kept demanding retired promises. Now detected by check_requirement_drift.py rather than quietly closed.

This is not a pilot. No human has used QAIA in their own work. One target, one API, no UI, 32 scenarios. Full campaign, protocol and limits →

Install

/plugin marketplace add QAIA-Project/QAIA
/plugin install qaia-core@qaia

Then describing what you want in plain language — "write the tests for this user story" — is enough: generate carries the journey step by step and stops wherever a human has to decide.

/plugin install qaia-playwright@qaia      # runnable Playwright tests, contract probing, defect reports
/plugin install qaia-score@qaia           # judge: scoring and release gate

plugins/qaia-core/CATALOGUE.md maps "I want to X → use Y" across all 7 skills. Worked examples with their real output are in examples/.

What separates it, and what does not

Applying ISTQB techniques is no longer a differentiator — QASkills.sh alone publishes ~380 competently written MIT skills, and Claude-Code-native competitors (Agentic QE Fleet, QA Orchestra) overlap directly. Three things do separate QAIA, all verifiable by a stranger in five minutes:

This is the beginning of the README. Read the rest at the original. Read the full README ↗