Explore what generative AI can do, local or in the cloud.
Docs · Tutorials · How-to · Reference · Examples · Notebooks
AIMU is a comprehensive generative AI framework that makes it easy to understand what AI models and systems are capable of, and to learn how they work while you use them. It gives you a single provider-agnostic interface across text, images, audio, and speech, autonomous agents and code-controlled workflows, and small composable utilities for tools, memory, prompt tuning, evaluations, and benchmarking. All of it in plain Python you can read, so nothing a model does is hidden behind the library.
Whether you need vision input, autonomous tool use, image generation, audio generation, or text-to-speech, the call is one line:
aimu.chat("What's in this photo?", model="...", images=["photo.jpg"])
aimu.agent("...", tools=builtin.web).run("Search the web and summarize today's AI news")
aimu.generate_image("a watercolor fox in a snowy forest", model="...")
aimu.generate_audio("a lo-fi hip-hop beat with soft piano", model="...")
aimu.generate_speech("Hello, world!", model="...")
Composition happens by passing objects to constructors. Conversation state is a list[dict] you can print and edit. Provider-specific details adapt at request time and never leak into your code.
Why AIMU
Most of the difficulty in learning what these models can do is the layer between you and them. AIMU tries to be the thinnest useful one. Swapping providers is a string change, so the difference you observe is the models' and not your harness's. A streamed run is labelled by phase, so you can watch reasoning, tool calls, and output separately. Conversation state is a list[dict] you can print. A capability the library can't deliver raises instead of quietly doing something else. When a model surprises you, the surprise should be the model's.
Six principles get it there: plain Python, plain data (OpenAI message dicts only), composability through uniform interfaces, progressive disclosure of capabilities, direct paths for common tasks, and apparent failures. The reasoning behind each, and the patterns each one excludes, lives on the design principles page.
A curated model catalog is part of that design. Every model AIMU ships knows what it can do: whether it calls tools, reasons, accepts images or audio, and enforces structured output, along with the sampling profile its model card specifies, declared once and shared across every provider that serves those weights. Naming a model AIMU has no spec for raises rather than running on guessed capabilities, since a wrong guess surfaces later as a confusing runtime failure. A model outside the catalog is still reachable: build its spec and pass it directly, or, for an OpenAI-compatible server, declare the capabilities inline in the model string, which can also carry a custom endpoint.
Install
pip install aimu[all]
Or install only what you need. AIMU's extras fall into two kinds:
Provider backends (the model/inference providers you use):
aimu[ollama]: local Ollama modelsaimu[anthropic]: Anthropic Claudeaimu[openai_compat]: OpenAI + Gemini cloud and OpenAI-compatible local servers (also enables OpenAI TTS speech and transcription STT)aimu[hf]: HuggingFace (text + diffusers image + audio + TTS speech + local embeddings)aimu[google]: Google Gemini image generation (Nano Banana)aimu[llamacpp]: local GGUF models via llama-cpp-python
Capabilities (cross-provider features, independent of which provider you pick):
aimu[web]: the Streamlit/Gradio chat apps + the personal-assistant web UIaimu[tuning]: the prompt-tuning subsystem + the evals benchmark harness (pulls pandas/tqdm)aimu[evals]: DeepEval metric adaptersaimu[a2a]: Agent-to-Agent protocol interop
aimu[all] is every provider backend plus every capability.
One temporary exception: [evals] can't share an environment with [hf] for now due to a click<8.4. HF transformers requires a newer version click. Install aimu[evals] in its own environment until deepeval relaxes that pin.
The PyPI release can lag behind what's availble in Github. Anything under Unreleased in the CHANGELOG ships on main but is not yet on PyPI. To use those features, install from source until the next release:
pip install "aimu[all] @ git+https://github.com/saxman/aimu@main"
Start exploring
There is a path through this, roughly in order:
This is the beginning of the README. Read the rest at the original. Read the full README ↗
