[!NOTE]
New in 1.0: decision models. Next to LLMs, ContextGem now works with decision models, which answer closed questions (yes/no, one of your labels, a level of a scale) with the probability of each possible answer, fast and at a low cost. A model group sends each aspect and concept to the model that suits it, and any model can fall back on others: see LLMs and decision models.
Upgrading from 0.x? Some classes and parameters are renamed or removed, some defaults have changed, and Python 3.10 is no longer supported. Follow the migration guide, or point a coding agent at llms.txt, the index of the documentation.
ContextGem is a free, open-source framework that does the work around the models for you: the prompts, the data models, the source references and the justifications, with minimal code.
Why ContextGem
Reliable extraction from documents usually means writing prompts, designing validation models, mapping outputs back to the source, orchestrating multi-step pipelines and tracking usage across models. ContextGem has all of this built in, in one declarative pipeline that you can reuse and serialize.
Read more on the motivation behind ContextGem.
LLMs and decision models
ContextGem works with two kinds of models. You describe what to extract in the same way for both, and each aspect and concept goes to the model that suits it.
| LLM | Decision model |
|---|
| How it answers | Writes the answer, which ContextGem validates into typed items | Gives the probability of each possible answer, and writes no text |
| What it extracts | Aspects and every type of concept, from text and images | Aspects, and Boolean, label and rating concepts, from text |
| With each item | References to paragraphs or sentences, and a justification on request | The probabilities of all the possible answers, and a decision threshold that you set for the model, or for each aspect and concept |
| Use it for | Open values, and answers that take reasoning | Closed questions in volume, where a fast, low-cost answer is enough |
| Models | Any cloud or local LLM through LiteLLM: OpenAI, Anthropic, Google, Azure OpenAI, xAI, Ollama, LM Studio and more | TypeSafe Jev, OpenAI Decisions API, Microsoft Foundry and more, cloud or local |
| Read more | Supported LLMs, configuration, extraction methods | Decision models |
Choosing a model. ContextGem is tested with hosted LLMs such as gpt-6-luna (OpenAI) and claude-haiku-5-5 (Anthropic), and with decision models such as Jev (TypeSafe). A smaller LLM, such as many that run locally, may not follow the detailed extraction instructions: check it on a sample of your documents, and see the troubleshooting guide if it falls short. Before you give questions to a decision model, compare it with an LLM on a sample of your documents: see when to use a decision model.
Quick start
uv add contextgem # or: pip install -U contextgem
uv add "contextgem[docling]" # with Docling, to convert PDF, DOCX and other files
Extract with an LLM
An LLM extracts aspects and values of any type. Below, it extracts two things from a contract: an aspect, the payment terms, which comes back as the paragraphs on payments, and a concept, the anomalies, which span the whole document and which retrieval-based approaches can miss. Each anomaly comes with a justification and the sentence it was found in.
This is the beginning of the README. Read the rest at the original. Read the full README ↗
