⛓️🛠️ ChainForge
An open-source visual environment for battle-testing prompts to LLMs.
ChainForge is a data flow prompt engineering environment for analyzing and evaluating LLM responses. It enables rapid-fire, quick-and-dirty comparison of prompts, models, and response quality that goes beyond ad-hoc chatting with individual LLMs. With ChainForge, you can:
- Query multiple LLMs at once to test prompt ideas and variations quickly and effectively.
- Compare response quality across prompt permutations, across models, and across model settings to choose the best prompt and model for your use case.
- Setup evaluation metrics (scoring function) and immediately visualize results across prompts, prompt parameters, models, and model settings.
- Use AI to streamline this entire process: Create synthetic tables and input examples with built-in genAI features, or supercharge writing evals by prompting a model to give you starter code.
Read the docs to learn more. ChainForge comes with a number of example evaluation flows to give you a sense of what's possible, including 188 example flows generated from benchmarks in OpenAI evals.
ChainForge is built on ReactFlow and Flask.
For user-curated resources and learning materials, check out the 🌟Awesome ChainForge repo!
Table of Contents
Installation
You can install ChainForge locally, or try it out on the web at https://chainforge.ai/play/. The web version of ChainForge has a limited feature set. In a locally installed version you can load API keys automatically from environment variables, write Python code to evaluate LLM responses, or query locally-run models hosted via Ollama.
To install Chainforge on your machine, make sure you have Python 3.10 or higher, then run
pip install chainforge
Once installed, do
chainforge serve
Open localhost:8000 in a Google Chrome, Firefox, Microsoft Edge, or Brave browser.
Security: ChainForge can run Python code on your computer, so its server only accepts requests from ChainForge's own page. It refuses requests from other websites, and requests addressed to any name other than localhost or 127.0.0.1. If you reach ChainForge by another name or address (on a server, say, or from another machine), add it: chainforge serve --host 0.0.0.0 --allowed-hosts your.server.name.
Retrieval-augmented generation (RAG)
ChainForge's RAG nodes -- Upload, Chunk, Retrieval and Rerank -- work in the
browser with no extra installation: documents, chunking, keyword retrieval,
semantic search and reranking all run client-side on small models fetched the
first time you use them.
For the server-side RAG stack as well -- hosted embedding providers, persistent
LanceDB vector stores, TF-IDF and the larger rerankers -- install the extra:
pip install chainforge[rag]
A note on FAISS
The faiss_vector_store retrieval method works whenever faiss is importable,
but ChainForge deliberately does not install it for you. The FAISS pip wheels
bundle their own OpenMP runtime, which collides with any other library that
brings one: installed alongside PyTorch, the combination crashes the process
outright, and upstream has closed every report of this without a fix. If you
need FAISS, install it yourself:
pip install faiss-cpu
ChainForge will then limit OpenMP to a single thread at startup to keep the two
runtimes from killing each other, and say so when it does. That makes embedding
slower, which is why it only happens when FAISS is actually present. Set
OMP_NUM_THREADS yourself to override it.
You can set your API keys by clicking the Settings icon in the top-right corner. If you prefer to not worry about this everytime you open ChainForge, we highly recommend that save your OpenAI, Anthropic, Google, etc API keys and/or Amazon AWS credentials to your local environment. For more details, see the How to Install.
Run using Docker
Quick start with Docker Compose (recommended):
docker compose up -d
Access ChainForge at http://localhost:8000
To open it from another machine, tell ChainForge the name or IP address you use, e.g. -e CHAINFORGE_ALLOWED_HOSTS=your.server.name. Requests addressed to other names are refused.
For detailed Docker documentation including architecture support, environment variables, and CI/CD setup, see docker/README.md.
Supported providers
- OpenRouter (models from many providers, including image models, with one API key)
- OpenAI
- Anthropic
- Google Gemini
- DeepSeek
- HuggingFace (Inference Providers and dedicated Endpoints)
- Together.ai
- Ollama API (locally-hosted models)
- Microsoft Azure OpenAI Endpoints
- Amazon Bedrock-hosted on-demand inference, including Anthropic Claude 3
- ...and any other provider through custom provider scripts!
This is the beginning of the README. Read the rest at the original. Read the full README ↗
