VerifyAX Claude Plugins

A Claude Code plugin marketplace for VerifyAX — the agent evaluation and verification platform from Conscium.

What is VerifyAX?

VerifyAX is a platform for testing and evaluating AI agents. You register an agent (an A2A or REST endpoint), generate scenarios that exercise specific skills, run your agent through them as simulations, and get back scored evaluations plus full transcripts — so you can benchmark, regression-test, and verify agent behaviour instead of eyeballing it.

These plugins bring that whole loop into Claude Code. Instead of clicking through the console or hand-writing API calls, you describe what you want ("register this agent and run an empathy eval") and Claude drives VerifyAX for you.

Who this is for

• You already have (or are evaluating) a VerifyAX account and want to drive it from Claude Code rather than the web UI.
• You're building or testing an AI agent and want repeatable, scored evals wired into your dev workflow.

If you've never heard of VerifyAX, start at verifyax.com to see what the platform does first — these plugins are a client for it, not a standalone tool. You'll need an account and an API key (below) before they do anything useful.

Plugins

Plugin	Description
verifyax-api	Drive the VerifyAX REST API programmatically (a skill) — register agents, generate scenarios, trigger simulations, fetch evaluations. Best for writing code.
verifyax-mcp	The same actions as native MCP tools Claude calls directly — register agents, generate scenarios, evaluate, inspect results. Best for conversational, no-code use. Installs @verifyax/mcp-server.
verifyax-claude-agent	Evaluate your own Claude Code agent — exposes it over A2A so VerifyAX can run scenarios against it (wraps the local claude CLI, guided by a skill). Best for testing a Claude agent you're building. Needs Python + a tunnel (set up for you automatically); reuses verifyax-api.

Not sure which? Pick verifyax-mcp for conversational workflows (Claude calls the tools for you); verifyax-api when you want Claude to write scripts against the API. Pick verifyax-claude-agent when the thing you want to evaluate is itself a Claude Code agent — it exposes yours for VerifyAX to score (and drives verifyax-api under the hood).

Where each plugin runs

These plugins target Claude Code. Two of them need a local machine, so they don't install and run in Claude.ai chat or Cowork — plugins with local MCP servers don't load there, and neither do plugins that shell out to local binaries.

Plugin	Claude Code	Claude.ai chat / Cowork
verifyax-api	✅	✅ — it's a pure skill (also uploadable as a bundle, see below)
verifyax-mcp	✅	❌ as a plugin — but see the connector route below
verifyax-claude-agent	✅	❌ — needs local Python, the claude CLI, and a tunnel

[!TIP] On Claude.ai you don't need the verifyax-mcp plugin — use the hosted endpoint directly. The plugin can't run there because it launches a local MCP server, but that constraint doesn't apply to the hosted one. In Settings → Connectors → Add custom connector, point at https://mcp.verifyax.com/mcp, choose No sign-in, and add a request header Authorization with the value Bearer sk-ver-api-.... All 12 tools appear, with per-tool approval controls. See the MCP server README for details. Not yet tested in Cowork.

On an Enterprise plan your admin may block this entirely — org policy can restrict which marketplaces users may add, mark individual plugins unavailable, or disable local MCP servers outright. If /plugin marketplace add does nothing, that's usually why; ask your Claude administrator.

Before you start

1. Create a VerifyAX account (or sign in) at the VerifyAX console.
2. Get an API key — Settings → API Keys in the console. Keys look like sk-ver-api-... and the full secret is shown only once at creation, so copy it then.
3. Make sure your workspace has credits. Generating scenarios and running simulations consume credits; operations fail with a payment error if the balance is empty.

Install

Add the marketplace, then install whichever plugin you want:

/plugin marketplace add verifyax/verifyax-plugins-claude
/plugin install verifyax-api@verifyax-plugins             # the skill
/plugin install verifyax-mcp@verifyax-plugins             # the MCP server
/plugin install verifyax-claude-agent@verifyax-plugins    # expose YOUR Claude agent for eval (auto-installs verifyax-api)

How you provide the API key depends on the plugin:

• verifyax-mcp prompts for the key when you enable the plugin and stores it securely.

• verifyax-api reads it from the VERIFYAX_API_KEY environment variable:

  export VERIFYAX_API_KEY=sk-ver-api-...
  

  The skill reads the key from this variable; it will never ask you to paste it into a chat.

• verifyax-claude-agent uses the same VERIFYAX_API_KEY (via verifyax-api), and additionally needs the claude CLI plus its adapter's Python deps (pip install -r plugins/verifyax-claude-agent/adapter/requirements.txt). It exposes your Claude agent over a public tunnel it sets up for you, then runs the eval. Run it with /verifyax-claude-agent:connect-to-verifyax — see its README for the guided flow, the two modes (tools-off / tools-on), and exactly what gets sent to VerifyAX.

Quickstart: zero to your first evaluation

Once a plugin is installed and your key is set, just describe the goal in plain language — Claude chains the steps (register → generate scenario → simulate → evaluate → fetch results) for you. A first run, end to end:

"Using VerifyAX: register my A2A agent at https://my-agent.example.com, generate one info_exchange scenario tagged empathy, run my agent against it, and show me the evaluation scores when it's done."

Claude will register the agent, create the scenario (async — it polls until ready), trigger the simulation, wait for it to complete, and return the scores. From there you can iterate:

• "Register this A2A agent on VerifyAX and run a quick interview-style eval against it."
• "List my failed simulations from the last week and show the error_details for each."
• "Generate a batch of 5 info_exchange scenarios with the empathy and active_listening tags."

💡 Runs cost credits — ask Claude to "estimate the credits before running" and it'll preview the cost first.

Key concepts

A quick glossary so the examples above make sense:

Term	What it is
Agent	The AI endpoint you're testing — an A2A or REST service you register with VerifyAX.
Scenario	A test environment your agent is run through. Two types: info_exchange (multi-agent, the default) and interview (1-to-1).
Skill tag	A label (e.g. empathy, active_listening) that targets what a scenario measures.
Simulation run	One execution of an agent against a scenario. Produces a transcript.
Evaluation	The scoring of a completed run against the scenario's ground truth.
Job	An async handle for long-running work (scenario creation, simulation, evaluation): PENDING → PROCESSING → COMPLETED / FAILED / CANCELLED.
Credits	Workspace spend consumed by generation, simulation, and evaluation.

The pipeline: Register agent → Create scenario → Trigger simulation → Evaluate → Fetch results.

Learn more

• Choosing between the plugins, the skill and the SDK — and which Claude surface supports what: using-verifyax-with-claude.md, the canonical guide. It's kept in the repo that changes the code, so it tracks versions and client support rather than drifting from them.
• Platform & sign-up: verifyax.com · console.verifyax.com
• Company: Conscium
• Full API reference: the canonical OpenAPI contract at console.verifyax.com/openapi.yaml (the verifyax-api skill carries the workflow + behavioural rules and fetches the contract for exact endpoint shapes).

Update

Installed plugins do not update themselves. Claude Code keeps the version you installed until you refresh the marketplace and reinstall — a machine that installed in May can still be running the May build months later. Refresh with:

/plugin marketplace update verifyax-plugins
/plugin install verifyax-api@verifyax-plugins             # repeat for each plugin you use

Check what you actually have with /plugin, and compare against the current versions listed under Versioning below.

Also available as a Claude.ai skill

If you use Claude.ai (not Claude Code), download verifyax-api.zip (or browse all releases), then in Claude.ai go to Customize → Skills, click +, choose + Create skill, and select Upload a skill. Same SKILL.md, different wrapper.

[!NOTE] Each release carries the same bundle under two names — verifyax-api.zip and verifyax-api.skill. They are byte-identical, and both upload successfully (tested), so take either; .zip just matches the extension Claude.ai's own docs name. Note that Settings → Capabilities → Skills only views and toggles skills you already have — uploading happens under Customize → Skills.

Link to the version tag, never to /releases/latest/download/.... Each plugin here is tagged and released independently, so latest is whichever plugin shipped most recently — today that is verifyax-claude-agent, which carries no .zip at all, and a latest/download link for the skill returns 404. Use the pinned URL above and bump it when the skill version changes.

Building the .skill bundle (maintainers)

The bundle attached to each release is built from the plugin's skill directory with:

scripts/build-skill.sh            # → dist/verifyax-api.zip (+ .skill, same bytes)
scripts/build-skill.sh <plugin> <skill>   # for any other plugin/skill

The bundle is a zip whose top level is a single <skill>/ folder containing SKILL.md (and any resource files) — the layout Claude.ai requires. The script emits it twice: .zip, the extension Claude.ai's uploader documents, and .skill, kept so links to older release assets keep working. The release workflow attaches both automatically on a verifyax-api-v* tag. Build output lives in dist/ (gitignored).

Versioning

Each plugin pins an explicit version in its plugin.json and is versioned independently. Users only receive updates when we bump the version, so we bump on every release. Current versions: verifyax-api 0.3.0, verifyax-mcp 0.3.5, verifyax-claude-agent 0.1.2. See CHANGELOG.md for release notes.

License

Apache-2.0. See LICENSE.

Contributing

Found a bug, an outdated endpoint, or have an idea for a new plugin (workbench helpers, scenario authoring, etc.)? Open an issue and we'll take it from there. The maintainers handle changes internally, so external pull requests aren't accepted — issues are the best way to reach us. See CONTRIBUTING.md for the model, and SECURITY.md to report a vulnerability privately.