any-llm

Communicate with any LLM provider using a single, unified interface. Switch between OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, Ollama, and more without changing your code.

Documentation | otari.ai | Try the Demos | Contributing

Quickstart

pip install 'any-llm-sdk[mistral,ollama]'

export MISTRAL_API_KEY="YOUR_KEY_HERE"  # or OPENAI_API_KEY, etc
from any_llm import completion
import os

# Make sure you have the appropriate environment variable set
assert os.environ.get('MISTRAL_API_KEY')

response = completion(
    model="mistral-small-latest",
    provider="mistral",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

That's it! Change the provider name and add provider-specific keys to switch between LLM providers.

Coming from LiteLLM? Your API keys and environment variables carry over unchanged. Install the SDK with extras for the providers you need, then update your import and model strings:

pip install 'any-llm-sdk[openai,anthropic]'  # or [all] for everything

# before
from litellm import completion
response = completion(model="openai/gpt-4o", messages=[...])

# after
from any_llm import completion
response = completion(model="openai:gpt-4o", messages=[...])

See Supported Providers to map your existing model strings.

That's the full migration — no proxy, no extra config.

Installation

Requirements

• Python 3.11 or newer
• API keys for whichever LLM providers you want to use

Basic Installation

Install support for specific providers:

pip install 'any-llm-sdk[openai]'           # Just OpenAI
pip install 'any-llm-sdk[mistral,ollama]'   # Multiple providers
pip install 'any-llm-sdk[all]'              # All supported providers

See our list of supported providers to choose which ones you need.

Using an OpenAI-compatible gateway or local server that isn't listed? You don't need a dedicated provider entry: see Custom OpenAI-compatible Endpoints.

Setting Up API Keys

Set environment variables for your chosen providers:

export OPENAI_API_KEY="your-key-here"
export ANTHROPIC_API_KEY="your-key-here"
export MISTRAL_API_KEY="your-key-here"
# ... etc

Alternatively, pass API keys directly in your code (see Usage examples).

Otari Gateway

For budget management, API key management, usage analytics, and multi-tenant support, see mozilla-ai/otari.

Why choose any-llm?

• Simple, unified interface - Single function for all providers, switch models with just a string change
• Developer friendly - Full type hints for better IDE support and clear, actionable error messages
• Leverages official provider SDKs - Ensures maximum compatibility
• Stays framework-agnostic so it can be used across different projects and use cases
• Battle-tested - Powers our own production tools (any-agent)

Usage

any-llm offers two main approaches for interacting with LLM providers:

Option 1: Direct API Functions (Recommended for Bootstrapping and Experimentation)

Recommended approach: Use separate provider and model parameters:

from any_llm import completion
import os

# Make sure you have the appropriate environment variable set
assert os.environ.get('MISTRAL_API_KEY')

response = completion(
    model="mistral-small-latest",
    provider="mistral",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Alternative syntax: Use combined provider:model format:

response = completion(
    model="mistral:mistral-small-latest", # <provider_id>:<model_id>
    messages=[{"role": "user", "content": "Hello!"}]
)

Option 2: AnyLLM Class (Recommended for Production)

For applications that need to reuse providers, perform multiple operations, or require more control:

from any_llm import AnyLLM

llm = AnyLLM.create("mistral", api_key="your-mistral-api-key")

response = llm.completion(
    model="mistral-small-latest",
    messages=[{"role": "user", "content": "Hello!"}]
)

When to Use Which Approach

Approach	Best For	Connection Handling
Direct API Functions (completion)	Scripts, notebooks, single requests	New client per call (stateless)
AnyLLM Class (AnyLLM.create)	Production apps, multiple requests	Reuses client (connection pooling)

This is the beginning of the README. Read the rest at the original. Read the full README ↗