Lexguard

The problem

You're evaluating an AI agent's reply: polite or rude, confident or overclaiming, clean or full of leaked secrets. The obvious move is to grep for a few words, and a single word list breaks fast: a reply that says "could you please fix the fucking bug" contains "please" and swears in the same breath. One list can't tell those apart — it either misses the swearing or scores the reply as polite.

Lexguard is still just word matching. The difference is a second list per concept: words that rule it back out. A check returns whether the concept holds, is simply absent, or is present but denied by the wording around it.

from lexguard import Politeness

print(Politeness("could you send this over when you get a sec?"))
#> True
print(Politeness("send me the report"))
#> False
print(Politeness("could you please fix the fucking bug"))
#> False

The last two both fail, but for different reasons: the second reply never mentions politeness, the third does and then undercuts it. .verdict(text).reason is how you tell those apart:

from lexguard import Politeness

print(Politeness.verdict("could you please fix the fucking bug").reason)
"""
politeness wording present but denied by: "fucking"
fix: add a courteous phrase (please, thanks, could you) and don't undercut it with sarcasm or profanity
"""

Paste your own text into the playground to watch every lexicon score it live in the browser, no install needed.

How it works

A Lexicon is a named set of words and phrases that signal a concept (indicates), the words that rule it back out (rules_out), and a one-sentence remedy for a hit (fix). It's matched by plain substring — no model in the loop, so a check is instant and deterministic. A set ships in the box already, tuned for agent transcripts — slop, sycophancy, leaked secrets, overclaimed confidence, politeness, and more; run lexguard to list every one. To extend one, edit its list: lexguard <name> prints it as source to paste into your own module. See Writing a lexicon.

A Lexicon is callable — Politeness(text) is shorthand for Politeness.matches(text), itself built on a three-valued .signal() (present / absent / denied), with .denied() alongside it for the third case. Reach for those when you just want a plain value. .verdict(text) is the richer pass/fail-plus-.reason built on top, and it's the same call every integration below wraps, whether that's an eval framework or a guardrail.

One lexicon, either turn

A lexicon detects a raw semantic act, not a speaker, so the same lexicon reads whichever turn you point it at. Correction is the act of taking back a previous statement: run it over the user's message to catch them revising their own request, or over the agent's message to catch the agent walking back its own output.

from lexguard import Correction

print(Correction("scratch that, use a set instead"))
#> True
print(Correction("on second thoughts that won't compile, let me change that"))
#> True

The built-ins are grouped into facets of meaning (time, intent, tone, epistemics, safety, and more), never by who is speaking, because who speaks is a wiring choice you make when you hand a turn to a check.

What a match means

Most lexicons are checked for absence — Slop, Confidential, Rudeness are things you don't want, so a match is the failure. A few, like Confirmation and Politeness, are checked for presence instead: they set fail_when_neutral=True, so silence is the failure.

matched?	fail_when_neutral=False (default)	fail_when_neutral=True
yes	✘ FAIL — e.g. Slop, found	✔ PASS — e.g. Confirmation, said
no	✔ PASS — e.g. Slop, clean	✘ FAIL — e.g. Confirmation, silent

See Writing a lexicon for the full explanation.

Install

uv add lexguard

The core has no dependencies. Each integration is its own extra:

uv add "lexguard[pydantic-evals]"
uv add "lexguard[deepeval]"
uv add "lexguard[inspect-ai]"
uv add "lexguard[pydantic-ai-harness]"

To use it as a dev tool without adding it to any particular project, install it as one:

uv tool install lexguard

Lexguard is pre-1.0, so any release may break things: word lists, score and label names, and the API itself. Pin an exact version if you depend on scores staying comparable across upgrades.

Using it without an evals framework

.verdict() is a plain call over a string — nothing here needs Dataset, Case, or pydantic-evals in general, and you get the diagnostic reason for free:

from lexguard import Confidential


def guard(reply: str) -> str:
    verdict = Confidential.verdict(reply)
    if not verdict.passed:
        raise ValueError(verdict.reason)
    return reply


print(guard("send it over, all good"))
#> send it over, all good

A reply that actually leaks something raises with the full diagnosis attached — see Failures tell you what to change below for what that reason text looks like.

Running it inside pydantic-evals

LexguardEvaluator, from lexguard.integrations.evals.pydantic_evals, wraps a lexicon as an evaluator, for when you want the same check running as part of a Dataset alongside everything else.

This is the beginning of the README. Read the rest at the original. Read the full README ↗