any-llm-ts

Use multiple LLM providers through one typed TypeScript interface.

An independent TypeScript port inspired by mozilla-ai/any-llm.

Documentation

any-llm-ts is a thin, framework-independent layer over official provider SDKs. It gives applications one API for chat completions, Messages, streaming, tools, structured output, embeddings, model discovery, the OpenAI Responses API, batches, reranking, images, moderation, audio, and provider-hosted files without requiring a hosted proxy.

The package uses official provider SDKs where native translation is required. OpenAI-compatible providers share a data-driven adapter, so switching providers is usually one string change.

Installation

npm install any-llm-ts

Node.js 20 or newer is required.

Quick start

import { completion } from "any-llm-ts";

const response = await completion({
  provider: "openai",
  model: "gpt-4.1-mini",
  messages: [{ role: "user", content: "Hello!" }],
});

console.log(response.choices[0]?.message.content);

Set the provider's conventional environment variable first, such as OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY, or MISTRAL_API_KEY. You can also pass apiKey explicitly.

The combined provider:model form is useful for small scripts:

const response = await completion({
  model: "anthropic:claude-sonnet-4-5",
  messages: [{ role: "user", content: "Explain subgrid in one paragraph." }],
});

Reusable clients

Create a client once when your application makes multiple requests. The underlying SDK client and its connection pool are reused.

import { AnyLLM } from "any-llm-ts";

const llm = AnyLLM.create("mistral");

const first = await llm.completion({
  model: "mistral-small-latest",
  messages: [{ role: "user", content: "Give me a project name." }],
});

const second = await llm.completion({
  model: "mistral-small-latest",
  messages: [{ role: "user", content: "Give me another one." }],
});

Streaming

stream: true changes the inferred return type to AsyncIterable<ChatCompletionChunk>.

const stream = await completion({
  provider: "groq",
  model: "llama-3.3-70b-versatile",
  messages: [{ role: "user", content: "Write a haiku about TypeScript." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta.content ?? "");
}

Errors raised while consuming a stream are normalized just like errors raised while creating it.

Tool calling

Tools use the widely supported OpenAI function-tool shape. The Anthropic and Gemini adapters translate tools, tool choices, assistant tool calls, and tool results in both directions.

const response = await completion({
  provider: "anthropic",
  model: "claude-sonnet-4-5",
  messages: [{ role: "user", content: "What is the weather in Kolkata?" }],
  tools: [
    {
      type: "function",
      function: {
        name: "get_weather",
        description: "Get current weather for a city",
        parameters: {
          type: "object",
          properties: { city: { type: "string" } },
          required: ["city"],
        },
      },
    },
  ],
});

console.log(response.choices[0]?.message.toolCalls);

OpenAI-compatible endpoints

Use any unlisted OpenAI-compatible gateway or local server without registering it globally:

const llm = AnyLLM.createOpenAICompatible({
  name: "company-gateway",
  apiBase: "https://llm.example.com/v1",
  apiKey: process.env.COMPANY_LLM_API_KEY,
});

const response = await llm.completion({
  model: "internal-model",
  messages: [{ role: "user", content: "Hello" }],
});

Set requiresApiKey: false for a keyless local endpoint.

Providers

The port registers the same 53 provider names as the tracked Python source revision.

Adapter	Providers
OpenAI and OpenAI-compatible	openai, azureopenai, plus the data-driven compatible-provider registry
Anthropic	anthropic, azureanthropic, vertexaianthropic
Google Gen AI	gemini, vertexai
AWS	bedrock, sagemaker
Other native SDKs and protocols	azure, cohere, github, huggingface, meta, mistral, otari, together, voyage, watsonx

Registry metadata is intentionally conservative. Inspect capabilities at runtime instead of assuming every provider implements every OpenAI endpoint:

const metadata = AnyLLM.getProviderMetadata("deepseek");
console.log(metadata.capabilities);

const providers = AnyLLM.getAllProviderMetadata();

Metadata also exposes the Python project's verified/community tier and prompt-cache-key policy. Provider registration reflects API compatibility, not a claim that every provider, model, region, and operation is continuously integration-tested with live credentials.

Gateway contracts

Provider descriptors expose stable IDs, adapter and package provenance, capability metadata, and a closed credential-configuration contract. OpenAI and Anthropic publish versioned API-key schemas; providers whose configuration is not yet safely self-describing remain discoverable with configuration.status === "unavailable".

Use the second completion argument for controls that must never enter the Provider JSON payload:

import { completion, getProviderDescriptors } from "any-llm-ts";

const controller = new AbortController();
const dispatches = [];

const response = await completion(
  {
    provider: "openai",
    model: "gpt-4.1-mini",
    messages: [{ role: "user", content: "Hello!" }],
  },
  {
    signal: controller.signal,
    retryPolicy: "none",
    onDispatch: (evidence) => dispatches.push(evidence),
  },
);

console.log(getProviderDescriptors());

retryPolicy: "none" sets the qualified Provider SDK's per-request retry count to zero. Dispatch evidence records that the adapter crossed into the Provider SDK; it does not claim that the Provider received or completed the request. A signal aborted before that boundary produces no dispatch event. For descriptors declaring providerOptions: "normalized_fields_win", Provider extensions may add fields but cannot replace normalized completion fields.

Other operations

The reusable client exposes:

await llm.responses({ model, input });
await llm.embedding({ model, input });
await llm.listModels();
await llm.imageGeneration({ model, prompt });
await llm.imageEdit({ model, image, prompt });
await llm.transcription({ model, file });
await llm.speech({ model, input, voice });
await llm.moderation({ input });
await llm.messages({ model, maxTokens, messages });
await llm.rerank({ model, query, documents });
await llm.createBatch({ endpoint, inputFilePath });
await llm.retrieveBatch(batchId);
await llm.cancelBatch(batchId);
await llm.listBatches();
await llm.retrieveBatchResults(batchId);
await llm.uploadFile({ file });
await llm.listFiles();
await llm.retrieveFile({ fileId });
await llm.downloadFile({ fileId });
await llm.deleteFile({ fileId });

Stateless camel-cased helpers with the same names are exported from the package. An unsupported operation rejects with UnsupportedOperationError. Provider-specific request fields can be added through providerOptions and SDK constructor fields through clientOptions. Use maxRetries on AnyLLM.create() to set the SDK retry count uniformly (0 disables retries).

messages() uses native Messages support where available and a normalized completion compatibility layer elsewhere.

Errors

Provider SDK failures are converted into a common hierarchy:

• AuthenticationError
• InvalidRequestError
• RateLimitError, including retryAfter from Retry-After or retry-after-ms
• ModelNotFoundError
• ContextLengthExceededError
• ContentFilterError
• ProviderError, UpstreamProviderError, and GatewayTimeoutError
• ProviderFileNotFoundError

Each AnyLLMError retains the original error as cause and exposes provider-independent statusCode, code, param, errorType, and provider fields where available.

Custom adapters

Extend BaseProvider, then register a factory. This keeps provider-specific translation isolated while preserving the common facade.

import { registerProvider } from "any-llm-ts";

// CompanyProvider extends BaseProvider and implements metadata and completion().
// companyMetadata must include the complete, versioned Provider descriptor.
registerProvider("company", (options) => new CompanyProvider(options), {
  metadata: companyMetadata,
});

See PORTING.md for the architectural analysis, what was deliberately preserved, and where TypeScript-specific choices differ from the Python project.

The published docs are at any-llm-ts.anuragroy.dev. The Fumadocs site is maintained in apps/docs. Run npm run docs:dev on Node.js 22 or newer to preview it locally.

Development

npm install
npm run check

npm run check runs Oxfmt verification, Oxlint, strict type-checking, unit tests with coverage gates, and the dual ESM/CommonJS build.

Attribution and license

This is an independent port, not an official Mozilla.ai project. It takes architectural inspiration from mozilla-ai/any-llm, which is licensed under Apache-2.0. See NOTICE and LICENSE.