OpenMarkX

About

OpenMarkX (NeurIPS 2026) is an extensible platform for watermarking open-source model and agent outputs. It learns a watermark into the weights of a model before release, tests the watermark under the edits a user can make to the weights, and detects it later with the developer's secret key. Language, vision-language, agent, image and physics models run through one command, openmarkx run, and methods are compared under the same edits with openmarkx bench.

OpenMark, the paper's method, trains the watermark policy jointly with the LLM, so the model learns the pattern the detector expects, and adds perturbation-aware training, so the watermark survives fine-tuning, merging, quantization and pruning. On LLaMA3-8B it reaches 0.92 TPR at 1% FPR, and across three open-source LLMs it lowers perplexity by 20.5% on average relative to Gu et al.-KGW.

Key Features

• 🔏 OpenMark — a watermark policy trained with the model (CAWP) and forgetting-perturbation training (FPL).
• 🧪 16 methods — OpenMark, 12 MarkLLM decoding watermarks learned into the weights, GaussMark, WMAR and Ripple.
• 🔧 Weight and text edits — fine-tuning, merging, quantization and pruning of the weights; rewriting of the text.
• 🌐 Beyond text — vision-language models, agents, image generators and physics surrogates.
• 🧩 Extensible — any Hugging Face causal LM by --base; a new method in one file; a new model behind an adapter.

Installation

git clone https://github.com/loopleap-lab/OpenMarkX.git
cd OpenMarkX
pip install -e .                    # images: pip install -e ".[vision]"
bash scripts/install_deps.sh        # MarkLLM at a pinned commit, for the markllm-* methods
python exp/prepare_data.py          # C4 text and held-out prompts (streams from Hugging Face)

OpenMarkX needs Python 3.10+ and a CUDA GPU; the Colab notebook runs on a free T4. Runs and keys go under ~/.openmarkx; back up the keys, since a lost key cannot be recovered.

How to Use OpenMarkX

Watermark a Language Model

1. Embed the watermark into a base model and freeze the detection threshold:

openmarkx embed --base llama-3.2-1b --watermark openmark    # prints {"release": RELEASE, "key": KEY}
openmarkx calibrate KEY --base llama-3.2-1b

Publish RELEASE; keep KEY. --watermark openmark-frozen (the policy kept fixed, no FPL) is about three times faster per step for a first run.

2. Edit the release as a user would:

openmarkx modify RELEASE --edit finetune:steps=300 --out edited/

3. Detect the watermark in suspect texts, or verify a suspect model:

openmarkx generate edited/ --prompts prompts.jsonl --out texts.jsonl
openmarkx detect texts.jsonl --key KEY                      # one p-value per text
openmarkx verify edited/ --key KEY --prompts prompts.jsonl  # one verdict for the model

prompts.jsonl holds one {"text": ...} per line (a .txt file with one prompt per line also works). The same in Python:

import openmarkx as omx

release, key = omx.embed("llama-3.2-1b", "openmark")
key = omx.calibrate(key, "llama-3.2-1b")
edited = omx.modify(release, "finetune:steps=300")
hits = omx.detect(omx.generate(edited, ["The history of the city begins"]), key)

Run a Demo

One config runs a whole experiment: it learns the mark into the weights, checks outputs with the key, applies the config's edits and writes report.json. A stopped run picks up where it stopped; --smoke runs a few steps.

openmarkx run configs/demos/geopt.yaml       # physics: GeoPT aircraft fields
openmarkx run configs/demos/taming.yaml      # images: Taming Transformers
openmarkx run configs/demos/medgemma.yaml    # medical VQA: MedGemma 1.5
openmarkx run configs/demos/agent.yaml       # a ReAct agent on Llama-3.1-8B-Instruct
openmarkx detect field.npz --model geopt --key ~/.openmarkx/runs/demos/geopt/release

Benchmark Methods

openmarkx bench configs/arena/1b.yaml

scores methods under quantization, pruning, merging and fine-tuning and writes a leaderboard of your own runs; LEADERBOARD.md has the paper's results.

FAQ

Demo	Method	What it needs	Imprint time of the site's run
geopt, geopt_colearn	ripple	a GeoPT checkout in OMX_GEOPT_DIR with checkpoints/craft_geopt_8layers.pt; the AirCraft data in $OMX_DEMO_DATA/geopt/aircraft_npys	geopt: 7 s (edits: 2 min)
taming	wmar	a WMAR checkout in OMX_WMAR_DIR; the Taming checkpoint directory in TAMING; the vision extra	55 min
medgemma	openmark-frozen	the gated google/medgemma-1.5-4b-it; data from exp/med/prepare.py and exp/med/distill_data.py; the vision extra	1 h 51 min
agent	openmark-frozen	data from exp/agent/prepare.py and exp/agent/distill_data.py	36 min

Each demo needs one GPU. Runs go to ~/.openmarkx/runs/demos/<name>/; publish its release/, which does not hold the secret key.

A causal language model needs only --base with a Hugging Face repo id or a local directory. The paper's LLaMA-3-8B loads as --base meta-llama/Meta-Llama-3-8B.

A field model, such as a neural PDE surrogate, plugs into the geopt adapter through one small class named in the config:

# run.yaml
model: {name: geopt, factory: "my_field:MyField", ckpt: null, data: null}
scheme: {name: ripple, params: {eps: 0.05}}
imprint:
  phases:
    - {steps: 400, lr: 1.0e-3, trainable: full}
detect: {fpr: 0.01, null: wrong_keys, n_null: 20}
edits: ["finetune:steps=20,lr=1e-4"]
quality: rel_l2
out: runs/my_field

PYTHONPATH=. openmarkx run run.yaml

The tutorial has a runnable my_field.py and walks through the GeoPT and Taming adapters.

以上为 README 的开头部分。其余内容请阅读原文。 查看完整 README ↗