Litefuse

Litefuse is an open source LLM engineering platform. It helps teams collaboratively develop, monitor, evaluate, and debug AI applications. Litefuse can be self-hosted in minutes and is battle-tested.

✨ Core Features

• LLM Application Observability: Instrument your app and start ingesting traces to Litefuse, thereby tracking LLM calls and other relevant logic in your app such as retrieval, embedding, or agent actions. Inspect and debug complex logs and user sessions. Try the interactive demo to see this in action.

• Prompt Management helps you centrally manage, version control, and collaboratively iterate on your prompts. Thanks to strong caching on server and client side, you can iterate on prompts without adding latency to your application.

• Evaluations are key to the LLM application development workflow, and Litefuse adapts to your needs. It supports LLM-as-a-judge, user feedback collection, manual labeling, and custom evaluation pipelines via APIs/SDKs.

• Datasets enable test sets and benchmarks for evaluating your LLM application. They support continuous improvement, pre-deployment testing, structured experiments, flexible evaluation, and seamless integration with frameworks like LangChain and LlamaIndex.

• LLM Playground is a tool for testing and iterating on your prompts and model configurations, shortening the feedback loop and accelerating development. When you see a bad result in tracing, you can directly jump to the playground to iterate on it.

• Comprehensive API: Litefuse is frequently used to power bespoke LLMOps workflows while using the building blocks provided by Litefuse via the API. OpenAPI spec, Postman collection, and typed SDKs for Python, JS/TS are available.

📦 Deploy Litefuse

Two ways to run Litefuse: use the managed cloud, or self-host on your own infrastructure (Local, VM, Docker, Kubernetes via Helm, or Terraform on AWS / GCP / Azure).

Litefuse Cloud

Managed deployment by the Litefuse team, generous free-tier, no credit card required.

Self-Host Litefuse

Run Litefuse on your own infrastructure:

• Local (docker compose): Run Litefuse on your own machine in 5 minutes using Docker Compose.

  # Get a copy of the latest Litefuse repository
  git clone https://github.com/litefuse/litefuse.git
  cd litefuse
  
  # Run the litefuse docker compose
  docker compose up
  

• VM: Run Litefuse on a single Virtual Machine using Docker Compose.

• Kubernetes (Helm): Run Litefuse on a Kubernetes cluster using Helm. This is the preferred production deployment.

• Terraform Templates: AWS, Azure, GCP

See self-hosting documentation to learn more about architecture and configuration options.

🔌 Integrations

Main Integrations:

Integration	Supports	Description
SDK	Python, JS/TS	Manual instrumentation using the SDKs for full flexibility.
OpenAI	Python, JS/TS	Automated instrumentation using drop-in replacement of OpenAI SDK.
Langchain	Python, JS/TS	Automated instrumentation by passing callback handler to Langchain application.
LlamaIndex	Python	Automated instrumentation via LlamaIndex callback system.
Haystack	Python	Automated instrumentation via Haystack content tracing system.
LiteLLM	Python, JS/TS (proxy only)	Use any LLM as a drop in replacement for GPT. Use Azure, OpenAI, Cohere, Anthropic, Ollama, VLLM, Sagemaker, HuggingFace, Replicate (100+ LLMs).
Vercel AI SDK	JS/TS	TypeScript toolkit designed to help developers build AI-powered applications with React, Next.js, Vue, Svelte, Node.js.
Mastra	JS/TS	Open source framework for building AI agents and multi-agent systems.
API		Directly call the public API. OpenAPI spec available.

Packages integrated with Litefuse:

Name	Type	Description
Instructor	Library	Library to get structured LLM outputs (JSON, Pydantic)
DSPy	Library	Framework that systematically optimizes language model prompts and weights
Mirascope	Library	Python toolkit for building LLM applications.
Ollama	Model (local)	Easily run open source LLMs on your own machine.
Amazon Bedrock	Model	Run foundation and fine-tuned models on AWS.
AutoGen	Agent Framework	Open source LLM platform for building distributed agents.
Flowise	Chat/Agent UI	JS/TS no-code builder for customized LLM flows.
Langflow	Chat/Agent UI	Python-based UI for LangChain, designed with react-flow to provide an effortless way to experiment and prototype flows.
Dify	Chat/Agent UI	Open source LLM app development platform with no-code builder.
OpenWebUI	Chat/Agent UI	Self-hosted LLM Chat web ui supporting various LLM runners including self-hosted and local models.

🚀 Quickstart

Instrument your app and start ingesting traces to Litefuse, thereby tracking LLM calls and other relevant logic in your app such as retrieval, embedding, or agent actions. Inspect and debug complex logs and user sessions.

1️⃣ Create new project

1. Create Litefuse account or self-host
2. Create a new project
3. Create new API credentials in the project settings

2️⃣ Log your first LLM call

The @observe() decorator makes it easy to trace any Python LLM application. In this quickstart we also use the Litefuse OpenAI integration to automatically capture all model parameters.

[!TIP] Not using OpenAI? Visit our documentation to learn how to log other models and frameworks.

pip install langfuse openai

LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_BASE_URL="https://litefuse.cloud"

from langfuse import observe
from langfuse.openai import openai # OpenAI integration

@observe()
def story():
    return openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "What is Litefuse?"}],
    ).choices[0].message.content

@observe()
def main():
    return story()

main()

3️⃣ See traces in Litefuse

See your language model calls and other application logic in Litefuse.

Public example trace in Litefuse

[!TIP]

Learn more about tracing in Litefuse or play with the interactive demo.

⭐️ Star Us

If Litefuse is useful to you, please consider giving the repo a star. It helps more people discover the project and motivates us to keep improving it.

💭 Support

Finding an answer to your question:

• Our documentation is the best place to start looking for answers. It is comprehensive, and we invest significant time into maintaining it. You can also suggest edits to the docs via GitHub.
• Litefuse FAQs where the most common questions are answered.
• Use "Ask AI" to get instant answers to your questions.

Support Channels:

• Ask any question in our public Q&A on GitHub Discussions. Please include as much detail as possible (e.g. code snippets, screenshots, background information) to help us understand your question.
• Request a feature on GitHub Discussions.
• Report a Bug on GitHub Issues.
• For time-sensitive queries, ping us via the in-app chat widget.

🤝 Contributing

Your contributions are welcome!

• Vote on Ideas in GitHub Discussions.
• Raise and comment on Issues.
• Open a PR - see CONTRIBUTING.md for details on how to setup a development environment.

🙏 About this fork

About this fork — Litefuse is a fork of langfuse/langfuse, and we are deeply grateful to the Langfuse team for the original work. Litefuse swaps the analytics backend to Apache Doris and maintains wire-protocol compatibility with the upstream Langfuse API, so the official langfuse / @langfuse/* SDKs and any Langfuse-compatible OpenTelemetry exporter work against Litefuse without code changes — only the host needs to be pointed at your Litefuse endpoint.

🥇 License

This repository is MIT licensed. See LICENSE for details.

Dependencies

We deploy this code base in Docker containers based on the Linux Alpine Image (source). You may find the Dockerfiles in web/Dockerfile and worker/Dockerfile.

🔒 Security & Privacy

We take data security and privacy seriously. Please refer to our Security and Privacy page for more information.

Telemetry

By default, Litefuse automatically reports basic usage statistics of self-hosted instances to a centralized server (PostHog).

This helps us to:

1. Understand how Litefuse is used and improve the most relevant features.
2. Track overall usage for internal and external (e.g. fundraising) reporting.

None of the data is shared with third parties and does not include any sensitive information. We want to be super transparent about this and you can find the exact data we collect here.

You can opt-out by setting TELEMETRY_ENABLED=false.