duckdb-ai: LLMs and embeddings in DuckDB SQL

Call large language models from SQL. Summarize, classify and filter rows, extract typed fields, embed text for semantic search, and ask questions about your tables, using local models (Ollama, llama.cpp, any OpenAI-compatible server) or hosted providers (OpenAI, Anthropic Claude, Google Gemini, OpenRouter, Databricks, Snowflake Cortex and more than 30 others).

Documentation · SQL reference · Provider setup · Cookbooks · Agent guide

SELECT ticket_id,
       ai_classify(body, ['billing', 'performance', 'other']) AS category,
       ai_summarize(body) AS summary
FROM support_tickets;

Contents

• Names at a glance
• Quickstart with a local model
• Connect a hosted provider
• Choose a function
• Examples, from simple to advanced
• Using the extension from an agent
• Supported providers
• Privacy, security and limits
• Build from source
• Documentation

Names at a glance

What	Name
Repository and project	duckdb-ai
Extension (INSTALL / LOAD)	ai
SQL functions	ai_*
Settings	duckdb_ai_* (for example SET duckdb_ai_provider = 'ollama')
Secret type	TYPE duckdb_ai

The extension is published in the DuckDB community extension catalog and works anywhere DuckDB can load community extensions: the CLI, Python, and other clients.

[!NOTE] This README describes the source on main. The community package can lag behind it. Check your installed version and function list (see step 1 below) before relying on a newer function, and see the changelog for what changed when.

Quickstart with a local model

This path needs no API key and sends no data off your machine.

1. Install and check the installed API

INSTALL ai FROM community;
LOAD ai;

-- Installed version
SELECT extension_version
FROM duckdb_extensions()
WHERE extension_name = 'ai';

-- Available functions (makes no model calls)
SELECT function_name, function_type
FROM duckdb_functions()
WHERE starts_with(function_name, 'ai_')
ORDER BY function_name;

LOAD ai; is needed once per connection. Loading the extension never calls a model by itself.

main and releases from 0.9.0 target DuckDB 2.0. The 0.8.x releases are the DuckDB 1.5.6 line.

2. Start Ollama and pull a model

Install Ollama, then in a terminal:

ollama serve                    # skip if Ollama is already running
ollama pull qwen3.8:27b         # chat model
ollama pull nomic-embed-text    # embedding model (optional)

3. Call it from SQL

LOAD ai;

SET duckdb_ai_provider = 'ollama';
SET duckdb_ai_model = 'qwen3.8:27b';
SET duckdb_ai_embedding_model = 'nomic-embed-text';

SELECT ai_complete('Describe DuckDB in one sentence.');

Without a model setting, provider := 'ollama' uses llama3.2, so pull that model or set one as shown. For ai_decide, also run ollama pull nimble (Ollama 0.35 or later).

The extension calls Ollama at http://localhost:11434. Set OLLAMA_HOST, or pass base_url := ..., if it runs elsewhere. For llama.cpp, vLLM, LM Studio or LiteLLM, see local and self-hosted setup.

Connect a hosted provider

Keep API keys out of SQL. Put the key in the DuckDB process environment (or in a secret manager that injects it), then create a named secret that holds only the provider and model:

export OPENAI_API_KEY='...'

LOAD ai;

-- If you ran the Ollama quickstart in this session, clear its settings first:
-- session settings outrank the provider and model stored in a secret.
RESET duckdb_ai_provider;
RESET duckdb_ai_model;

CREATE OR REPLACE SECRET openai_ai (
    TYPE duckdb_ai,
    AI_PROVIDER 'openai',
    MODEL 'gpt-5.6-luna'
);

SELECT ai_complete(
    'Describe DuckDB in one sentence.',
    secret := 'openai_ai',
    max_tokens := 64
);

Every provider follows the same pattern: an environment variable for the key, a secret or session setting for provider and model. The provider guides list the variable name, base URL and model IDs for each one. Hosted calls send your input to that provider and may incur charges.

How settings combine

For each call, the extension picks the provider and model from the first place that sets them:

1. Named options on the call: provider := ..., model := ...
2. A CREATE EXTERNAL MODEL profile passed with profile := ...
3. Session settings: the family setting (duckdb_ai_completion_model, duckdb_ai_task_model, duckdb_ai_embedding_model, duckdb_ai_sql_assistant_model), then duckdb_ai_provider and duckdb_ai_model
4. The AI_PROVIDER and MODEL stored in a TYPE duckdb_ai secret
5. Provider environment variables, such as OPENAI_MODEL, then the provider default

A secret always supplies the API key, but its model only applies when nothing above sets one. ai_decide is the exception: it ignores duckdb_ai_model and uses a decision model (see typed decisions).

This is the beginning of the README. Read the rest at the original. Read the full README ↗