Jev-Style
JevStyle is an open-source toolkit for running small decision models locally. Classify text, route requests, and score defined options with probabilities for each answer. Use it to route support tickets, decide when to call a larger model, and review receipts or screenshots with the image-capable build.
Maintained by Chaoliang Yan (lawrence3699). For local text classification in Python, start with the classification guide. To choose between a trained classifier, embedding-based routing, a general LLM, and typed decision models, read the local decision-model guide.
Try text and images online · Run locally · Copy a working recipe · Website
Give the model text or JSON, optional images, and typed questions. It returns yes/no probabilities, a distribution over choices, or a rating. Your application decides when to accept the result and when to ask a person or another model.
Guides and evaluation evidence
Quick Start
Try it in the browser
The 2B Space runs Jev-Style-2B-Decision-v4 (text and images), and the 0.8B Space runs the 0.8B. There is nothing to install.
Run it locally
You'll need Python 3.10 or newer. Install one backend for your machine:
# Apple silicon: local text decisions
pip install -U "jev-style[mlx]"
# NVIDIA GPU, CPU, or image input (also works on Apple MPS)
pip install -U "jev-style[torch]"
Start the smallest model and open the local Playground:
jev-style serve
The first start downloads the 0.8B weights (about 1.5 GB); later starts reuse them. In a second terminal, make one decision:
jev-style decide "I was charged twice" --url http://127.0.0.1:8765 --choice "Which team?::billing,shipping,returns"
Choose a model
| I want to… | Start this server | Images? | Approximate weight download |
|---|
| Try local text routing with a small model | jev-style serve | No | 1.5 GB, default 0.8B |
| Use the latest 2B for text on Apple silicon | jev-style serve --release 2b --backend mlx | No | 3.76 GB default; 2 GB at 8-bit |
| Read receipts or screenshots | jev-style serve --release 2b --backend torch | Yes | 4.43 GB |
| Keep using the previous 2B text model | jev-style serve --release 2b-v3 | No | Depends on backend; see Models |
These are weight-file sizes, not RAM or VRAM requirements. The 1.27 GB 2B GGUF file is text-only and needs the custom scorer. Stock Ollama, LM Studio and mlx_lm.generate cannot produce these decision scores. For images on a Mac, explicitly select --backend torch; automatic selection otherwise prefers text-only MLX.
On an NVIDIA GPU, jev-style serve --release 2b --backend torch --dtype bfloat16 runs the 2B v4 the way it was measured on the Decision Index, with CUDA graphs.
--release works the same for decide, download and mcp --model; JEV_STYLE_RELEASE=2b sets it for every command and for the Python client.
To install the command line tool on its own, use uv: uv tool install "jev-style[all]" on Apple silicon, uv tool install "jev-style[torch]" elsewhere. The MCP server is part of every install since 0.3.0 (jev-style[mcp] still works).
Recipes
Clone this repository to run the three complete examples. Each prints JSON and makes no changes to tickets or payments.
| Task | What you can run | What you learn |
|---|
| Support-ticket routing | python examples/recipes/ticket_router.py | Route high-probability tickets; send ambiguous ones to human review. |
| Fallback to another model | python examples/recipes/fallback_router.py --fallback-url http://127.0.0.1:8766 | Call a second systemone-compatible server only when needed; measure coverage and end-to-end latency. |
| Receipt review | python examples/recipes/receipt_review.py | Send a bundled synthetic receipt to the 2B v4 PyTorch server and flag it for manager review. |
The text examples include 12 hand-written synthetic tickets for checking the workflow. They are not a benchmark. Thresholds are illustrative; evaluate on your own held-out data before automating decisions. No cost saving or accuracy improvement is assumed.
Use Semantic Router
The local Semantic Router example
connects DecisionRouter to the running JevStyle server without an external API key.
Install semantic-router==1.1.1 in your environment, then run:
python examples/semantic_router_local.py "Where is my parcel?"
It proposes a route or returns a review step. The example threshold is illustrative;
measure it on your own labelled requests before using the result for automation.
API examples
Open http://127.0.0.1:8765 for the Playground and the demos: agent action approval, Snake, and Chinese and 51 languages. In another terminal, send a ticket:
curl -s localhost:8765/v1/systemone -H 'content-type: application/json' -d '{
"state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
"questions": {
"department": {"type": "choice", "instructions": "Which team should handle this?",
"criteria": {"returns": "Exchanges, refunds, wrong or damaged items",
"shipping": "Delivery status, delays, lost packages",
"billing": "Charges, invoices, payment problems"}},
"escalate": {"type": "noul", "instructions": "Does this need urgent human attention?"},
"frustration": {"type": "score", "instructions": "How frustrated is the customer?",
"criteria": ["Calm", "Frustrated", "Very angry"]}
}}'
This is the actual response from the 0.8B (the default release) with MLX on an Apple M1 Max, with probabilities rounded:
{
"model": "jev-style-0.8b-decision-v3",
"answers": {
"department": { "type": "choice", "choice": "billing", "confidence": 0.46,
"probabilities": { "returns": 0.07, "shipping": 0.29, "billing": 0.64 } },
"escalate": { "type": "noul", "noul": 0.45 },
"frustration": { "type": "score", "score": 1.28, "confidence": 0.36,
"legend": { "0": "Calm", "1": "Frustrated", "2": "Very angry" },
"probabilities": { "0": 0.07, "1": 0.57, "2": 0.35 } }
},
"usage": { "input_tokens": 250, "state_tokens": 25, "output_tokens": 0 },
"latency_ms": 194.0,
"backend": "mlx"
}
The ticket raises both a billing problem and a late delivery, and the probabilities show it: billing 0.64, shipping 0.29. That is why the model returns probabilities rather than a single label. Your code can act on the confident answers and hand the rest to a person.
Use it from Python
For a script, one line is enough. The first call loads the model in-process (the 0.8B, or the release in JEV_STYLE_RELEASE), or uses the server in JEV_STYLE_URL if you set it:
import jev_style
jev_style.classify("Where is my parcel? It was due Monday.", ["billing", "shipping", "tech"])
# {'label': 'shipping', 'confidence': ..., 'probabilities': {'billing': ..., 'shipping': ..., 'tech': ...}}
For several questions about one text, and to choose the engine yourself:
from jev_style import JevStyle, choice, noul, score
js = JevStyle.from_pretrained("chaoliangUNSW/Jev-Style-2B-Decision-v4-MLX") # the 2B v4 in-process; the repo picks the backend
# js = JevStyle.from_pretrained("chaoliangUNSW/Jev-Style-0.8B-Decision-v3-MLX") # the 0.8B
# js = JevStyle(release="2b") # the 2B v4, backend chosen for this machine
# js = JevStyle() # the default release (0.8B)
# js = JevStyle(base_url="http://127.0.0.1:8765") # or a running server
out = js.decide("I was charged twice. Please fix this ASAP.", {
"billing": noul("This ticket is about billing."),
"tone": choice("What is the customer's tone?", ["calm", "frustrated", "angry"]),
"urgency": score("How urgent is this ticket?", ["can wait", "this week", "today"]),
})
print(out["answers"]["billing"]["noul"], out["answers"]["tone"]["choice"])
from_pretrained takes the main repo (PyTorch), -MLX (precision="8bit" for the 2.00 GB 2B or 0.80 GB 0.8B weights) or -GGUF (quant="Q4_K_M", needs the release's scorer, jev-score-v2 for the 2Bs and jev-score for the 0.8B, see Backends).
Images go in the state by reference, with the image itself under assets (2B v4, PyTorch build):
from jev_style import JevStyle, image, noul
js = JevStyle.from_pretrained("chaoliangUNSW/Jev-Style-2B-Decision-v4") # PyTorch: text and images
out = js.decide({"receipt": "r1", "policy": "Meals over 50 EUR need a manager approval."},
{"approval": noul("Does this receipt need a manager approval?")},
assets={"r1": image("receipt.png")}) # PNG or JPEG, 280 to 1,568 px per side
print(out["answers"]["approval"]["noul"])
The client is not tied to this model: JevStyle(base_url=...) works with any server that implements POST /v1/systemone, and a client written for another systemone-compatible server can call http://127.0.0.1:8765 with any API key string.
From the shell
jev-style decide "Refund still missing after 3 weeks" --url http://127.0.0.1:8765 \
--choice "Which team?::billing,shipping,returns" --noul "The customer is angry"
# without --url (or JEV_STYLE_URL) it loads its own copy of the model (add --release 2b for the 2B v4)
Release notes and capabilities
New in 0.6.0: Jev-Style-2B-Decision-v4, the first Jev-Style model that reads images. On the Decision Index 0.3 public suite it scores 44.37, up from 34.05 for the 2B v3 on the board, and 57.55 on the public Vision suite (12,210 rows); on the 231 public JevBench v1.4.1 items it scores 74.9 % (v3: 73.6 %). All three are single self-runs with the official tools, not official board entries. It takes 32,768 tokens per call (v3: 25,600) and reads photos, screenshots, receipts and charts sent as assets, the way TypeSafe's API takes them. --release 2b now loads it (--release 2b-v3 keeps the v3). It was trained in part on soft labels distilled from TypeSafe Jev 1.14 via its public API. Try it in your browser.
New in 0.4.0: CUDA graphs for the PyTorch backend on NVIDIA GPUs: on an RTX 5090 the median latency falls from 86.2 to 13.9 ms for the 2B v3 and from 41.6 to 11.3 ms for the 0.8B, with no top-1 answer changed on 4,992 calibration questions. And cascades, which send only the questions a small model is unsure of to a larger one, including the named cascade-9b: our 2B followed by the third-party JevK5-9B.
Jev-Style is a family of small decision models built on Qwen3.5. The current releases are Jev-Style-2B-Decision-v4 (reads images; 1.27 GB at 4-bit, text only), Jev-Style-2B-Decision-v3 (1.27 GB at 4-bit) and Jev-Style-0.8B-Decision-v3 (0.53 GB at 4-bit, the default). Every build and demo is in the v4 collection and the v3 collection. You give a model text or JSON (and, with the 2B v4, images) and some typed questions, and it returns a calibrated probability for every option in one forward pass. The server's API follows the public systemone request shape, so clients written for Jev-compatible servers can call your laptop instead.
This repository is the part that makes the model useful day to day:
jev-style serve: a local API with a Playground and demos. It picks MLX on Apple silicon and PyTorch on CUDA or CPU; llama.cpp is optional.- Six agent skills: install with one
npx skills add. They serve the model, call it, evaluate it on your own labels, replace LLM calls that only return a label, add a guard to Claude Code, and add the MCP tools.
- A Claude Code guard: a
PreToolUse hook where the local model checks every tool call before it runs and answers allow, ask or deny.
- An MCP server: tools
decide, noul, choice and score for Claude Code, Cursor, Codex and any other MCP client.
jev-style eval: measures accuracy and calibration on your own labelled data, and reports how many decisions you can automate at a 1, 5 or 10 % error budget.
No GPU, no API key and no training needed.
Highlights
- Three question types in one request. Yes/no (
noul), multiple choice (choice, up to 255 options) and ordered ratings (score, 2 to 10 levels). The model reads the text once and answers every question about it.
- Probabilities, not just labels. Each release ships temperatures fitted on held-out data, so your code can act on confident answers and send the rest to a person or an LLM.
- Long inputs. Up to 32,768 tokens per call with the 2B v4 (25,600 with v3). Nothing is truncated: an input that is too long is rejected with an error that says so.
- Images (2B v4). Photos, screenshots, receipts, charts and scanned forms in the state, sent as
assets the way TypeSafe's API takes them. The PyTorch build reads them; the MLX and GGUF builds read text only.
- Runs locally. About 0.15 to 0.2 s for a short request to the 0.8B with MLX on an M1 Max, after the first call. Nothing leaves your machine.
- 51 languages evaluated. Training covers 19 languages. On MASSIVE intent the 0.8B beats Laya's multilingual checkpoint in all 51 evaluated languages.
- Built for agents. The skills, the MCP tools and the guard all work with Claude Code, Codex, Cursor and other agents.
Models
Release (--release) | Parameters · smallest build | JevBench v1.4.1 public (231) | tweet_topic, zero-shot (1,693) | Context |
|---|
Jev-Style-2B-Decision-v4 (2b) | 2.2B · 1.27 GB (Q4_K_M, text) | 74.9 % | not run | 32,768 tokens, images |
Jev-Style-2B-Decision-v3 (2b-v3) | 1.9B · 1.27 GB (Q4_K_M) | 73.6 % | 82.2 % | 25,600 tokens |
Jev-Style-0.8B-Decision-v3 (0.8b, default) | 0.8B · 0.53 GB (Q4_K_M) | 64.1 % | 75.5 % | 25,600 tokens |
| Hosted Jev 1.13, for reference | – | 86.6 % | 79.3 % | – |
The 2B v4 JevBench number is one self-run with the official harness through this package's server (float32, Apple MPS; 95 % CI 68.9 to 80.0 %), not an official board entry. On the Decision Index 0.3 public suite the 2B v4 scores 44.37 (one complete self-run with the kit's own runner and scorer, 140,178 requests; v3 on the board: 34.05) and 57.55 on the public Vision suite (12,210 rows); the board's headline Full score comes from the maintainers' private tests, so these are not a board rank. Several benchmarks' official train splits are in its training data, so those results are not zero-shot; its model card gives every protocol, and the 2B Space runs it in the browser.
The 2B v3 numbers are single pre-declared runs with its GGUF F16 build; its model card gives the protocols and confidence intervals. On JevBench the 2B v3 is the highest among the Qwen3.5-2B-family systems on the v1.4.1 board (decider-2b 71.0 %, open-jev-zefan-2b 64.5 %), but the lead over decider-2b is inside the 95 % confidence interval, 42 of the 82 board systems score higher, and hosted Jev is well ahead. On tweet_topic the 2B's accuracy is above Jev's published number, but its macro-F1 is below (0.678 vs 0.694). The 2B was trained on a reduced data pool (60M tokens) and has no separate limit for the question and its options; everything counts toward the 25,600 tokens.
The 0.8B against Laya, on sets neither was trained on:
| Model | Banking77 (77 intents, never trained) | MASSIVE intent, 37 held-out languages | tweet_topic, zero-shot | JevBench v1.4.1 public (231) |
|---|
| Jev-Style-0.8B-Decision-v3 | 68.2 % | 65.5 % | 75.5 % | 64.1 % |
| Best official Laya checkpoint (0.8B, 1,024 tokens by default) | 49.2 % | 36.1 % | 63.2 % | 58.4 % |
These numbers are from the 0.8B model card, which gives the full protocol and confidence intervals. The Laya rows are its official checkpoints re-run on the same rows, except tweet_topic and JevBench, which use published numbers. The hosted Jev API has higher accuracy than the 0.8B on every one of these sets where its accuracy is published. Treat both releases as small local options, not replacements for the hosted model.
| Build | 2B v4 | 2B v3 | 0.8B | Used by |
|---|
| safetensors: 2B v4, 2B v3, 0.8B | 4.43 GB | 3.76 GB | 1.50 GB | --backend torch (CUDA, Apple MPS, CPU); the 2B v4 reads images here |
| MLX bf16 / 8-bit: 2B v4, 2B v3, 0.8B | 3.76 / 2.00 GB | 3.76 / 2.00 GB | 1.50 / 0.80 GB | --backend mlx (Apple silicon; auto picks it there) |
| GGUF F16 / Q8_0 / Q4_K_M: 2B v4, 2B v3, 0.8B | 3.78 / 2.01 / 1.27 GB | 3.78 / 2.01 / 1.27 GB | 1.52 / 0.81 / 0.53 GB | --backend gguf (llama.cpp through the release's scorer: jev-score-v2 for both 2Bs, jev-score for the 0.8B) |
Each build carries its own runtime file next to the weights. The server downloads a pinned revision and uses that file, so the answers here match what the model card documents. Stock llama.cpp, Ollama, LM Studio or mlx_lm.generate can load the weights but cannot produce the decision scores. The 2B MLX runtimes need mlx-lm 0.31.3 exactly, which jev-style[mlx] installs. Images need the 2B v4 PyTorch build: its MLX and GGUF files hold the language model only. For long documents on the 2B, use Q8_0 (the default) or F16 rather than Q4_K_M. Earlier 2B generations, for use in LM Studio or Ollama without this server: v1 GGUF (LM Studio, llama.cpp) and v2 GGUF (Ollama).
Agent Skills
The skills live in skills/. Each one tells a coding agent how to do one job from start to finish, and ends with a check that the job worked.
npx skills add lawrence3699/jev-style # pick skills interactively
npx skills add lawrence3699/jev-style --skill '*' -y # all six
| Skill | What your agent does with it |
|---|
jev-style-serve | Installs the CLI, picks the backend for your machine, starts the server, checks it with a real request, and can optionally start it at login. |
jev-style | Writes code that calls the model: request format, reading the probabilities, turning them into actions with thresholds, limits. |
jev-style-eval | Builds a labelled JSONL from your data and runs jev-style eval. It reports accuracy, Brier, ECE, how much you can automate at a 1, 5 or 10 % error budget, and a refitted temperature. |
jev-style-adopt | Scans your code for LLM calls that only return a label, yes/no or a rating, and rewrites them as typed questions. The LLM stays as the fallback below a confidence threshold. It runs in shadow mode, measures agreement, then switches. |
jev-style-guard | Installs the Claude Code guard hook. It recommends a dry run first, then shows how to tune thresholds and measure them on labelled calls. |
jev-style-mcp | Registers the MCP tools with Claude Code, Codex, Cursor, Claude Desktop or Windsurf, and checks that a tool call works. |
In Claude Code you can also install everything as plugins:
/plugin marketplace add lawrence3699/jev-style
/plugin install jev-style@jev-style # the six skills + MCP tools
/plugin install jev-style-guard@jev-style # the PreToolUse guard
Claude Code Guard
Before Claude Code runs a Bash, Write, Edit, Read, WebFetch or MCP call, jev-style guard sends the call to the local model. It asks whether the call is destructive, exfiltrates data, touches secrets or goes outside the project, plus a 0 to 4 risk score. It turns the answers into allow / ask / deny. Regex hard rules in code can only make a verdict stricter. If the server is down, times out or rejects the request, the verdict is ask, never a silent allow.
jev-style guard --check "git push --force origin main"
# decision: DENY (source: rule, 443 ms) destructive 0.79 ...
jev-style guard-replay --url http://127.0.0.1:8765 # the 49 bundled labelled tool calls
On the 49 bundled tool calls, which were written and labelled by hand, the default config agrees with the labels on 77.6 %. No call labelled deny was allowed. 2 of the 16 calls labelled ask were allowed. The model alone, with no hard rules, agrees on 61.2 %. Use it as a second line of defence, not as a sandbox. The skill covers installing, dry runs and tuning.
MCP Server
claude mcp add jev-style --scope user -- jev-style mcp # Claude Code
jev-style mcp is a thin stdio server that forwards to the running jev-style serve, so one copy of the model serves every client. It is also listed in the official MCP Registry as io.github.lawrence3699/jev-style (runs as uvx jev-style mcp; set JEV_STYLE_URL if your server is not on http://127.0.0.1:8765). It provides decide (several questions about one input), noul, choice, score and model_info. Setup for Codex, Cursor and Claude Desktop is in the skill.
Evaluate on Your Own Data
The best evidence is your own labelled examples. jev-style eval takes JSONL in the request format, with a label on each question:
jev-style eval examples/support_tickets.jsonl --url http://127.0.0.1:8765
question n acc brier ece automate @1% / 5% / 10% error
team 30 93.3% 0.077 0.123 90.0% (p>=0.65) / 93.3% (p>=0.57) / 100.0% (p>=0.44)
escalate 30 96.7% 0.107 0.125 80.0% (p>=0.71) / 100.0% (p>=0.54) / 100.0% (p>=0.54)
mood 30 73.3% 0.433 0.174 40.0% (p>=0.56) / 40.0% (p>=0.56) / 46.7% (p>=0.54)
The 30 example tickets were written by hand for this repository. They demonstrate the format; they are not a benchmark. Read automate @5%: 93.3% (p>=0.57) like this: if you act only when the top probability is at least 0.57, the model handles 93.3 % of tickets with at most 5 % errors among them, and everything else goes to a person or an LLM. The report also refits a temperature on half the rows and shows its effect on the other half. See the skill for building a proper evaluation set.
Compare engines
Repeat --server to run the same file through several engines: this package's local model, another build or quantisation, or any other server that implements POST /v1/systemone. The first one is the reference.
jev-style eval examples/support_tickets.jsonl --server bf16=local:mlx --server q8=http://127.0.0.1:8799
90 questions answered by every engine; the table and the differences use only those.
engine answered failed rows acc brier log loss ece automate @5% error
bf16 90 0 87.8% 0.206 0.369 0.125 80.0% (p>=0.54)
q8 90 0 87.8% 0.206 0.370 0.117 78.9% (p>=0.54)
difference to bf16 (95 % paired bootstrap over rows; brier: lower is better):
q8 accuracy +0.0 pts [+0.0, +0.0] brier +0.001 [-0.001, +0.002]
That run compares the MLX bf16 weights in-process with jev-style serve --precision 8bit on an Apple M1 Max. Engine specs are NAME=URL, NAME=local[:backend], NAME=hf:<repo id>, NAME=jevk5:<repo id or folder>, NAME=cascade:<name or file> or NAME=fake; --key NAME=ENV_VAR sends a bearer token from an environment variable and --model NAME=MODEL sets the request's model field. Every engine is scored on the same questions: if one engine rejects a row (too long, too many options), that row is dropped for all of them and counted under failed rows. --json PATH writes every answer.
API
POST /v1/systemone with {"state": text | object | array, "questions": {id: question}}, plus "assets" for images (2B v4):
type | criteria | Answer |
|---|
noul | optional {"true": "...", "false": "..."} | noul = P(true) |
choice | {option: description or null}, 1 to 255 options | choice, probabilities, confidence |
score | 2 to 10 levels, lowest first: "label" or {"label", "description"} | score = expected level index, legend, probabilities, confidence |
Choice confidence = (k · p_max − 1) / (k − 1) for k options: 0 when the probabilities are uniform, 1 when one option takes all of them. Score confidence is TypeSafe's ordinal formula, which also counts how far probability sits from the most likely level m: max(0, 1 − Σ pᵢ·|i − m| / MAD_unif), with MAD_unif the same average distance for an even spread. Noul answers carry no confidence. Images follow TypeSafe's API: a string value in state names an entry of assets ({"type": "image", "format": "png" | "jpg", "data_base64": ...}, or a multipart file part), the request carries TypeSafe-Danger-Zone-Images: true, and each image is a still PNG or JPEG of 280 to 1,568 px per side; a model that reads text only answers 400 images_not_supported. Bodies may be compressed with Content-Encoding: gzip, x-gzip or zstd (the official SDKs compress by default), and the request rules match TypeSafe's API. The other routes are GET /v1/models, GET /healthz and the Playground at /. Errors look like {"error": {"code", "message", "question"?}} and use HTTP 400 (max_tokens_exceeded, request rules), 413, 415, 422 (invalid_json, invalid_request, invalid_question), 401 (unauthorized), 404 or 500. Start the server with --api-key-env NAME to require Authorization: Bearer <key>. The full reference is skills/jev-style/reference.md.
Cascades
A cascade answers each question with the smallest model that is confident enough and sends only the rest to a larger one. Tier 1 answers every question in one call; a question whose confidence (k · p_max − 1) / (k − 1) (for noul |2 · P(true) − 1|) is below the tier's threshold, or that the tier failed on, goes to the next tier in one call with the same state. The top tier's answer is final; if it fails, the last answer that succeeded is used, and only if every tier failed is the error returned. Answers keep the usual schema and add tier and tier_model; timing.tiers reports each tier's calls, and GET /v1/models reports the cascade's id, its tiers and their thresholds.
--cascade (for serve, decide and download), JevStyle(cascade=...) and the eval engine NAME=cascade:... take either the name of a cascade that ships with the package or a cascade file. jev-style releases lists the named cascades with their thresholds, tiers and pinned revisions.
cascade-9b
cascade-9b (model id jev-style-cascade-9b) is a cascade of our 2B with a third-party model:
- Jev-Style-2B-Decision-v3 runs in-process (PyTorch, float32, with CUDA graphs) and answers every question;
- a question whose confidence is below one threshold τ goes to JevK5-9B v0.3.3 by alibiserikbay (Apache-2.0), which runs in the same process through its own runtime, allebee/jevk5, at a pinned revision. It runs the same code as
jevk5-serve, so it gives the same answers as JevK5-9B behind jevk5-serve.
τ = 0.75, the same for every question type, was frozen on a 4,992-question calibration set (sha256 580962e4b2bc86b32b42731eda42536f57321c7d1ca9ac1e3b970bf7dbd7e2d2) with a rule written down before any result: the lowest expected latency whose calibration accuracy stays within 1.0 point of JevK5-9B alone. Details, the calibration sources and every number: chaoliangUNSW/Jev-Style-Cascade-9B.
| cascade-9b | |
|---|
| JevBench public items (231, our run) | 87.9 % (203 / 231; JevK5-9B alone 203, the 2B alone 170) |
| Questions answered by the 2B | 53 % of JevBench items, 43 % of the calibration set |
| Calibration set accuracy | 73.1 % (JevK5-9B alone 74.1 %, the 2B alone 66.2 %) |
| Median latency per request (RTX 5090) | 29 ms on JevBench items, 38 ms on the calibration set |
We do not claim that cascade-9b is more accurate than JevK5-9B on its own; what the cascade changes is how many questions reach the 9B at all. JevK5-9B's model card says that part of its training labels came from OpenAI's GPT-6 Luna, generated under OpenAI's terms; check that those terms suit your use.
It needs an NVIDIA GPU with room for both models: on an RTX 5090 (32 GB) it used 29.7 GB after start-up and at most 31.8 GB while serving, with PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True. It also needs PyTorch and the jevk5 package, which is not on PyPI:
pip install "jev-style[torch]" "jevk5[fast] @ git+https://github.com/allebee/[email protected]"
jev-style download --cascade cascade-9b # both tiers; the JevK5-9B snapshot is checked against its SHA256SUMS
jev-style serve --cascade cascade-9b --backend torch --device cuda
The JevK5 tier refuses to load without the model's jevk5_config.json (its calibration temperatures), since JevK5 would otherwise run uncalibrated. From Python the same cascade is JevStyle(cascade="cascade-9b").
Your own cascades
A cascade file lists the tiers (smallest first) and one threshold per tier below the top, for example the 2B in-process and JevK5-9B behind its own jevk5-serve:
jevk5-serve --model alibiserikbay/JevK5-9B --port 8090 # on the GPU box (see allebee/jevk5)
jev-style serve --cascade examples/cascade/glue-2.json --backend torch --device cuda
Tier targets are local[:release][:backend], hf:<repo id>, jevk5:<repo id or folder> (JevK5 in-process, as in cascade-9b; needs the jevk5 package), http(s)://... ("protocol": "jevk5" for jevk5-serve) or fake; "max_options" keeps questions with more options away from a tier, and "verify" checks a tier's files (a JevK5 tier's SHA256SUMS, on by default for the pinned JevK5-9B). The same file works as JevStyle(cascade="glue-2.json"), jev-style decide --cascade and jev-style eval --server glue=cascade:glue-2.json. The thresholds in examples/cascade/ are placeholders, not calibrated values. jev_style.cascade.select() replays recorded per-tier answers offline with the same rules.
Backends
| Machine | Command |
|---|
| Apple silicon | jev-style serve (MLX bf16; add --precision 8bit for 0.8 GB, or --release 2b --precision 8bit for the 2B at 2.0 GB) |
| NVIDIA GPU | jev-style serve --backend torch (with CUDA graphs) |
| CPU only | jev-style serve --backend torch --device cpu |
| llama.cpp | build the release's scorer once with sh build_jev_score.sh /path/to/llama.cpp from its GGUF repo (jev-score for the 0.8B, jev-score-v2 for the 2B), then jev-style serve --backend gguf --scorer /absolute/path/printed/by/the/script --quant Q4_K_M (add --release 2b for the 2B) |
| Offline | jev-style serve --model-dir /path/to/a/downloaded/model/repo |
| No model (UI or plumbing work) | jev-style serve --fake (deterministic, meaningless answers) |
CUDA Graphs
On an NVIDIA GPU the PyTorch backend records CUDA graphs when it starts and then replays them, instead of launching every kernel of a call one by one. Under --cuda-graphs auto the v3 releases use them whenever they run on CUDA, and the 2B v4 runtime decides itself: on with --dtype bfloat16, off in float32, where its forward is compute-bound and the graphs gain little. --cuda-graphs on|off, JEV_STYLE_CUDA_GRAPHS=1|0 or JevStyle(..., cuda_graphs=True|False) overrides that. It only applies to the PyTorch backend on CUDA.
The 2B v4 on an A100 80GB in bfloat16, 750 Decision Index 0.3 public requests one at a time: median 16.4 ms with CUDA graphs, 71.5 ms without them (its model card has the protocol and the parity checks). Recording takes about a minute the first time.
The v3 releases, measured on an RTX 5090 in float32 on 4,992 calibration questions against the same runtime without graphs:
| Release | Median latency | Top-1 answers changed | Largest probability difference |
|---|
| 2B v3 | 86.2 → 13.9 ms | 0 | 0.0024 |
| 0.8B | 41.6 → 11.3 ms | 0 | 0.00086 |
For the v3 releases, recording the graphs adds about 7 to 11 s to start-up. Inputs longer than 4,096 tokens, and several questions over a long shared state, take the ordinary path. The graphs live in the PyTorch builds' runtime file, at the revisions this version pins; with an older runtime, auto loads it as before and --cuda-graphs on is refused.
Repository Layout
jev_style/ server, client, CLI, guard, MCP server, eval; web/ = Playground + demos
skills/ six agent skills (npx skills add lawrence3699/jev-style)
plugins/ Claude Code guard plugin (the root .claude-plugin/ is the marketplace)
examples/ labelled example data for jev-style eval; cascade/ = example cascade files
scripts/ cascade_smoke.py: a two-tier cascade on fake engines, prints one routed response
space/ source of the 0.8B Hugging Face Space
space-2b/ source of the 2B v4 Hugging Face Space
tests/ pytest suite, runs on the fake engine (no model download)
Community and contributing
Share a minimal failure case, ask a usage question, or show a workflow in GitHub Issues. Please remove private data and credentials. The contribution guide includes setup, test commands, and small starting tasks.
Development
git clone https://github.com/lawrence3699/jev-style && cd jev-style
uv venv && uv pip install -e ".[dev]"
uv run pytest -q # fake engine: no weights, no GPU
uv run jev-style serve --fake # UI work without the model
Acknowledgements and License
Code: Apache-2.0 (LICENSE). The weights are Apache-2.0 fine-tunes of Qwen3.5-2B and Qwen3.5-0.8B; the NOTICE in each model repository lists the changes. The typed-question convention follows Laya.
cascade-9b and jevk5: tiers use JevK5-9B by alibiserikbay and its runtime allebee/jevk5, both Apache-2.0. Neither is included in this package: the runtime is installed separately and the weights are downloaded from their own repository. Per its model card, part of JevK5-9B's training labels came from OpenAI's GPT-6 Luna under OpenAI's terms.
Not affiliated with, endorsed by or connected to TypeSafe or Jev. "Jev-Style" describes the kind of model: a small typed-decision model in a similar style. This Python package does not distribute TypeSafe model weights, TypeSafe code, or raw teacher API-response datasets. That statement describes the package contents, not the model's training inputs: Jev-Style-2B-Decision-v4 was trained in part on soft labels distilled from TypeSafe Jev 1.14 through its public API, as disclosed in its model card. Check the individual model card for each release's training sources. Not affiliated with Alibaba Cloud or the Qwen team, the Laya authors or the JevK5 author.
