The first full-featured Turkish decision model
English • Türkçe • Website
Decision models answer typed questions about a text (for example, which team should handle a ticket, whether it
contains a prompt injection, or how urgent it is) and return a probability for every option instead of
generated text. Existing decision models (Decision 3.0, Jev, Laya and others) were developed for English, and their
accuracy on Turkish is limited. Pusula is Decision 3.0 adapted to Turkish.
- Turkish first. vllm-sr/d3 (Decision 3.0) has been fine-tuned on Turkish
decision data from 20 domains, including banking, accounting and tax, law, health, e-commerce, logistics and
public services. Informal writing, typos, missing Turkish characters and mixed Turkish–English text are covered.
- Options defined at request time. Questions and their options are specified in each request.
- Three question types.
choice (one option), noul (yes/no) and score (ordered levels).
- Drop-in d3 replacement. The request and answer format of d3's
/v1/systemone is preserved.
- Built-in Turkish taxonomy and templates. 20 domains, 103 subdomains and 333 intents, together with 20
ready-made templates (topic and intent, 14 checks for messages and 4 for AI agents). Routing through an
institution's own topic list is also supported.
- Turkish decision benchmark. Pusula has been evaluated against the base Decision 3.0 model on independently
labeled Turkish test sets; the results are published with the model. The benchmark is available on
Hugging Face.
Status: v2.0 (31 October 2026). Weights: Pusula ·
Pusula Nano. Earlier versions are listed under Releases.
Models
| Model | Size | Runs on | Benchmark / gold set | Use it for |
|---|
| Pusula (Turkish Decision 3.0) | 26B: a 0.9 GB LoRA adapter on top of vllm-sr/d3 | GPU: ≥ 64 GB in BF16, ≥ 24 GB in 4-bit | 92.0 / 92.1 | Highest accuracy, for built-in and custom questions |
| Pusula Nano | 149M (TabiBERT cross-encoder) | CPU or any GPU | 83.2 / 91.3 | High volume and low latency on the built-in questions |
Both models accept identical requests.
Results
Accuracy (%). Evaluation labels were assigned blind and independently by two annotators (Claude Opus 5.5 and
Claude Sonnet 5.5); disagreements were adjudicated, and all labels follow the published
labeling policy. The real customer messages were labeled blind by Claude under the same
policy. Evaluation labels are used exclusively for evaluation and are never used as training labels.
In v2.0, the evaluation sets were extended to the 20 templates and labeled under the v2.0 labeling policy. Decision 3.0
and the 1.0 models were therefore evaluated again on the same sets, and their scores differ from those published with
v1.0. The MASSIVE and OffensEval scores of Decision 3.0 and Pusula 1.0 are those of the v1.0 release.
| Evaluation | Questions | Decision 3.0 | Pusula 1.0 | Pusula 2.0 | Pusula Nano 1.0 | Pusula Nano 2.0 |
|---|
| Real customer messages (500 held-out, licensed) | 7,500 | 84.0 | 88.5 | 90.6 | 64.0 | 89.9 |
| Long customer complaints (500 held-out, licensed) | 4,500 | 81.5 | 88.1 | 88.3 | 84.6 | 84.6 |
| Turkish Decision Benchmark (2,194 messages) | 19,861 | 88.5 | 92.6 | 92.0 | 82.7 | 83.2 |
| Gold test set (757 held-out messages) | 11,355 | 86.3 | 90.5 | 92.1 | 62.8 | 91.3 |
| Custom questions in unseen sectors | 400 | 92.8 | 91.2 | 93.5 | 50.5 | 55.5 |
Missing information (eksik_bilgi) | 99 | 98.0 | 96.0 | 98.0 | 58.6 | 53.5 |
Sales opportunity (satis_firsati) | 100 | 97.0 | 97.0 | 97.0 |
Gold test set, per question:
以上为 README 的开头部分。其余内容请阅读原文。 查看完整 README ↗
