Loggetta
Large models. Smaller machines.
Plan the run. Train the model. Keep the evidence.
Get started · Train on your data · Measured results · Docs · Hugging Face
Loggetta checks your hardware, chooses a supported configuration for a Mixture-of-Experts (MoE) model,
and runs QLoRA fine-tuning. It saves the plan and a JSON run report with memory use, timing, and checks
that the selected optimizations actually ran.
One install includes the runtime and GPU kernels.
Limits. Plans are estimates, not an out-of-memory guarantee. The released training path is single-GPU MoE
training on Linux with a supported NVIDIA CUDA GPU. Dense models are planned but not yet supported for training.
Dense plans assume identical decoder layers, and a model whose layers differ is refused. For a config that declares
head_dim per layer, such as gemma4, the refusal names each difference (the per-layer head_dim, layers that reuse
another layer's KV, linears whose shapes change).
Get started · Dense training
Get started
pip install loggetta
loggetta inspect
loggetta plan Qwen/Qwen3-30B-A3B --seq 2048
The plan shows where weights will live, estimated memory use, and why alternatives were rejected.
It checks the budget before downloading model weights. Estimates can miss; they are not an out-of-memory guarantee.
GPU training needs Linux, a supported NVIDIA CUDA GPU, and compatible PyTorch.
The released training path is single-GPU MoE training. Dense execution still requires --allow-development-executor:
its memory estimate has not yet passed a capacity reading. See Dense training. A serve plan can be planned and measured with execute; server launch uses the runtime separately.
The driver and the torch build must match. A torch wheel is built for one CUDA version, and that version sets the
oldest NVIDIA driver it runs on: CUDA 13.x needs driver 580 or later, CUDA 12.x 525 or later (Linux). pip install loggetta installs the default torch from PyPI, which can be built for a newer CUDA than your driver supports.
loggetta inspect shows the driver, the torch build and whether torch can use the GPU, and plan refuses a CUDA plan
that torch cannot run, with the reason. To install a matching torch, use a fresh virtual environment and install torch
first, from the index for a CUDA version your driver supports, then Loggetta:
python -m venv .venv && . .venv/bin/activate
pip install torch --index-url https://download.pytorch.org/whl/<build> # pytorch.org lists each build's index
pip install loggetta
loggetta inspect
Installing a different torch build over an existing one does nothing: pip counts the same torch version as already
satisfied whatever CUDA it was built for. Start from a fresh environment (or pip install --force-reinstall from the
build's index).
loggetta inspect also names the build for your driver, and loggetta inspect --torch-index
prints only its tag (for example cu126), with no torch installed yet. scripts/first_run.sh does the whole first run
in a new directory: it reads the driver, installs that torch first, then Loggetta, and runs inspect, plan and 10
training steps on OLMoE-1B-7B-0924-Instruct at a pinned revision. Each step's exit status and time go to first_run.json, and it ends trained or with a
named refusal.
New in 0.7.0
- A first install ends trained or with a named refusal.
loggetta inspect --torch-index names the torch build for
your driver before torch is installed, and scripts/first_run.sh runs the whole first install. A machine with no
working C compiler is refused before any download.
execute runs serve plans through experts4bit-qlora's paged engine and writes a receipt with the measured peaks
and, at one sequence, the decode step.- MoE adapters record the quantized base they were trained on (
backend-adapter/2, with an experts4bit-qlora that
defines it), and load_adapter refuses a base that differs.
- Receipts carry no local path and record the backend switches they ran under. A run whose device check took no
driver sample reads
UNMEASURED and exits 4, not OK.
New in 0.6.2
- A CUDA plan the installed torch cannot run is refused, with the fix. When the torch build needs a newer
NVIDIA driver than the machine has,
plan names both and gives the install order that works, instead of letting
execute fail; inspect shows the torch build and whether it can use the GPU.
- The Alpaca demonstration runs past row 284. Training without
--dataset skips Alpaca's 28 empty-output rows,
counts them in the report, and is pinned to one dataset revision; a dataset you supply is still checked strictly.
New in 0.6.1
- gemma4 is refused with its reasons instead of failing. On transformers 5.20, describing
google/gemma-4-E2B-it
raised; plan now refuses it by name, listing each layer difference the planner cannot price yet.
- A model with no MoE layers is refused by the MoE backend at admission, so the dense backend owns it when it
can describe it, and otherwise the plan gives each backend's own reason.
New in 0.6.0
- An AMD GPU is found and refused by name.
plan says "AMD GPU detected (…); not yet supported" instead of "no
GPU", and inspect shows the gfx arch.
- The grouped_nf4 MoE backward is priced once. With experts4bit-qlora 0.53.0 or later, the runtime's estimate
prices it, and Loggetta adds no line of its own.
- Dense training's frozen-parameter digest takes a rank-0 parameter; every existing digest is unchanged.
- A held-out check on the RTX 5090 found one setup over its plan; every over-plan run used a setting the planner
does not model (pad buckets off, or grouped-nf4-gemm before 0.42.0)
(HO1, with addendum).
Full list: CHANGELOG.
Train on your data
Custom datasets and reusable adapters arrived in 0.3.0:
pip install -U loggetta
loggetta train Qwen/Qwen3-30B-A3B \
--dataset ./data/train.jsonl --format text \
--seq 512 --micro-batch 1 --steps 20 --seed 42 \
--out runs/my-training --adapter-out adapters/my-adapter
Use a JSONL file with a text field and enough tokens for the run. Local JSON, CSV, Parquet and TXT files,
Hub datasets, Alpaca instructions and text-only chats are also supported.
You keep adapter tensors, tokenizer files, a manifest, and a run report. Data is validated and tokenized before
weights load. A plan that would read the dataset more than once is refused unless you pass --epochs N or
--repeat-data; existing adapter directories are never overwritten.
from loggetta import load_adapter
model = load_adapter("adapters/my-adapter", device="cuda")
By default, chat data trains only the assistant turns and Alpaca data only the response; plain text and --loss all
score every token. Adapters use the native runtime format, not PEFT or
optimizer-resume checkpoints. Save/reload passes real-adapter CPU tests; the full CUDA integration test remains unverified.
Training and reload guide
Keep control of the run
| Need | Command or option |
|---|
| Set a memory budget | --vram 12 --ram 64 (GiB) |
| Choose where experts live | --experts device or --experts host |
| Save a plan | loggetta plan MODEL --out plan.json |
| Run that plan | loggetta execute plan.json --out runs/ |
| Reuse measurements | loggetta plan MODEL --observations runs/ |
| Plan serving placement | loggetta plan MODEL --workload serve --context 4096 --concurrency 1 |
| Measure a serve plan | loggetta execute plan.json --arena ARENA --calib CALIB |
The held-out band. A training plan also prints what held-out rows observed for the same model, card and placement:
measured / estimate, "observed on N held-out rows from M machines". It is not a prediction, and a further machine may
fall outside it. Where an estimate ran low on a held-out row, the plan warns. See
the held-out band.
train combines planning and execution. Dense training can be planned and executed with the development opt-in.
Multi-GPU execution, server launch and throughput prediction remain outside the current command surface.
Measured results
The included experts4bit-qlora runtime and grouped-nf4-gemm kernels do the compute.
Their speed results below come from matched training runs, not planner benchmarks.
| Workload | Result |
|---|
| Qwen3-30B-A3B QLoRA · RTX 5090 | Unsloth spends 1.92× e4b's GPU time per step, and 2.80× its wall-clock time on an AMD EPYC 7713 host. Comparable held-out loss; Unsloth peaked lower (24.27 vs 26.16 GB). Result |
| Why two numbers | GPU time doesn't depend on the host. Unsloth runs about 14× e4b's CPU operations per step, so its wall-clock time grows on a slower host. Earlier wall-clock readings, before e4b's current defaults: 2.352× and 2.468×. |
| Planner memory check · Qwen3-30B-A3B · RTX 5090 | After calibration from earlier runs: 24.54 GiB estimated process peak, 24.34 GiB measured. One in-sample case, not a guarantee. In the MoE audit re-run with training plans no longer borrowing another model's reserve (evidence/2026-10-09-moe-plan-vs-driver-no-borrow), today's planner puts no RTX A2000 training plan under its measured peak, in sample (the replan section; older recorded plans were). Plan vs run · MoE audit |
The comparison used torch 2.12.1+cu130 and transformers 5.5.0 for both frameworks, with matched adapters, initialization, and tokens.
Results apply to those workloads and hosts. See the runtime's current results for newer packed-training work.
Three projects, one install
| Project | Job |
|---|
| Loggetta | Inspect, plan, run, and save reports |
| experts4bit-qlora | Load, fine-tune, serve, and offload model weights |
| grouped-nf4-gemm | Compute on packed weights and move expert data |
The packages can also be used independently. Architecture and ownership
Documentation
Accelerate also estimates model memory before downloading weights and provides broader training and distributed infrastructure.
Loggetta focuses on choosing a configuration for its MoE runtime, running it, and feeding measured results into the next plan.
No matched estimator-accuracy comparison has been run.
pip install -e ".[test]"
pytest tests/
CPU tests cover planning and handoff. GPU benchmarks and CUDA adapter integration are separate checks.
