About
OpenMarkX (NeurIPS 2026) is an extensible platform for watermarking open-source model and agent outputs. It
learns a watermark into the weights of a model before release, tests the watermark under the edits a user can make to the weights, and detects it later with the
developer's secret key. Language, vision-language, agent, image and physics models run through one command,
openmarkx run, and methods are compared under the same edits with openmarkx bench.
OpenMark, the paper's method, trains the watermark policy jointly with the LLM, so the model learns the pattern the
detector expects, and adds perturbation-aware training, so the watermark survives fine-tuning, merging, quantization
and pruning. On LLaMA3-8B it reaches 0.92 TPR at 1% FPR, and across three open-source LLMs it lowers perplexity by
20.5% on average relative to Gu et al.-KGW.
Key Features
- 🔏 OpenMark — a watermark policy trained with the model (CAWP) and forgetting-perturbation training (FPL).
- 🧪 16 methods — OpenMark, 12 MarkLLM decoding watermarks learned into the weights, GaussMark, WMAR and Ripple.
- 🔧 Weight and text edits — fine-tuning, merging, quantization and pruning of the weights; rewriting of the text.
- 🌐 Beyond text — vision-language models, agents, image generators and physics surrogates.
- 🧩 Extensible — any Hugging Face causal LM by
--base; a new method in one file; a new model behind an adapter.
Installation
git clone https://github.com/loopleap-lab/OpenMarkX.git
cd OpenMarkX
pip install -e . # images: pip install -e ".[vision]"
bash scripts/install_deps.sh # MarkLLM at a pinned commit, for the markllm-* methods
python exp/prepare_data.py # C4 text and held-out prompts (streams from Hugging Face)
OpenMarkX needs Python 3.10+ and a CUDA GPU; the Colab notebook
runs on a free T4. Runs and keys go under ~/.openmarkx; back up the keys, since a lost key cannot be recovered.
How to Use OpenMarkX
Watermark a Language Model
1. Embed the watermark into a base model and freeze the detection threshold:
openmarkx embed --base llama-3.2-1b --watermark openmark # prints {"release": RELEASE, "key": KEY}
openmarkx calibrate KEY --base llama-3.2-1b
Publish RELEASE; keep KEY. --watermark openmark-frozen (the policy kept fixed, no FPL) is about three times
faster per step for a first run.
2. Edit the release as a user would:
openmarkx modify RELEASE --edit finetune:steps=300 --out edited/
3. Detect the watermark in suspect texts, or verify a suspect model:
openmarkx generate edited/ --prompts prompts.jsonl --out texts.jsonl
openmarkx detect texts.jsonl --key KEY # one p-value per text
openmarkx verify edited/ --key KEY --prompts prompts.jsonl # one verdict for the model
prompts.jsonl holds one {"text": ...} per line (a .txt file with one prompt per line also works). The same in
Python:
import openmarkx as omx
release, key = omx.embed("llama-3.2-1b", "openmark")
key = omx.calibrate(key, "llama-3.2-1b")
edited = omx.modify(release, "finetune:steps=300")
hits = omx.detect(omx.generate(edited, ["The history of the city begins"]), key)
Run a Demo
One config runs a whole experiment: it learns the mark into the weights, checks outputs with the key, applies the
config's edits and writes report.json. A stopped run picks up where it stopped; --smoke runs a few steps.
openmarkx run configs/demos/geopt.yaml # physics: GeoPT aircraft fields
openmarkx run configs/demos/taming.yaml # images: Taming Transformers
openmarkx run configs/demos/medgemma.yaml # medical VQA: MedGemma 1.5
openmarkx run configs/demos/agent.yaml # a ReAct agent on Llama-3.1-8B-Instruct
openmarkx detect field.npz --model geopt --key ~/.openmarkx/runs/demos/geopt/release
Benchmark Methods
openmarkx bench configs/arena/1b.yaml
scores methods under quantization, pruning, merging and fine-tuning and writes a leaderboard of your own runs;
LEADERBOARD.md has the paper's results.
FAQ
| Demo | Method | What it needs | Imprint time of the site's run |
|---|
geopt, geopt_colearn | ripple | a GeoPT checkout in OMX_GEOPT_DIR with checkpoints/craft_geopt_8layers.pt; the AirCraft data in $OMX_DEMO_DATA/geopt/aircraft_npys | geopt: 7 s (edits: 2 min) |
taming | wmar | a WMAR checkout in OMX_WMAR_DIR; the Taming checkpoint directory in TAMING; the vision extra | 55 min |
medgemma | openmark-frozen | the gated google/medgemma-1.5-4b-it; data from exp/med/prepare.py and exp/med/distill_data.py; the vision extra | 1 h 51 min |
agent | openmark-frozen | data from exp/agent/prepare.py and exp/agent/distill_data.py | 36 min |
Each demo needs one GPU. Runs go to ~/.openmarkx/runs/demos/<name>/; publish its release/, which does not hold
the secret key.
A causal language model needs only --base with a Hugging Face repo id or a local directory. The paper's LLaMA-3-8B
loads as --base meta-llama/Meta-Llama-3-8B.
A field model, such as a neural PDE surrogate, plugs into the geopt adapter through one small class named in the
config:
# run.yaml
model: {name: geopt, factory: "my_field:MyField", ckpt: null, data: null}
scheme: {name: ripple, params: {eps: 0.05}}
imprint:
phases:
- {steps: 400, lr: 1.0e-3, trainable: full}
detect: {fpr: 0.01, null: wrong_keys, n_null: 20}
edits: ["finetune:steps=20,lr=1e-4"]
quality: rel_l2
out: runs/my_field
PYTHONPATH=. openmarkx run run.yaml
The tutorial has a runnable my_field.py and walks through the GeoPT and Taming adapters.
여기까지가 README 앞부분입니다. 나머지는 원문에서 읽을 수 있습니다. README 전문 보기 ↗
