🧰 AI Agent Service Toolkit

A full toolkit for running an AI agent service built with LangGraph, FastAPI and Streamlit.

It includes a LangGraph agent, a FastAPI service to serve it, a client to interact with the service, and a Streamlit app that uses the client to provide a chat interface. Data structures and settings are built with Pydantic.

This project offers a template for you to easily build and run your own agents using the LangGraph framework. It demonstrates a complete setup from agent definition to user interface, making it easier to get started with LangGraph-based projects by providing a full, robust toolkit.

🎥 Watch a video walkthrough of the repo and app

Overview

Try the app!

Quickstart

Run directly in python

# At least one LLM API key is required
echo 'OPENAI_API_KEY=your_openai_api_key' >> .env

# uv is the recommended way to install agent-service-toolkit, but "pip install ." also works
# For uv installation options, see: https://docs.astral.sh/uv/getting-started/installation/
curl -LsSf https://astral.sh/uv/0.13.5/install.sh | sh

# Install dependencies. "uv sync" creates .venv automatically
uv sync --frozen
source .venv/bin/activate
python src/run_service.py

# In another shell
source .venv/bin/activate
streamlit run src/streamlit_app.py

Run with docker

echo 'OPENAI_API_KEY=your_openai_api_key' >> .env
docker compose watch

Architecture Diagram

Key Features

1. LangGraph Agent and latest features: A customizable agent built using the LangGraph framework. Implements the latest LangGraph v1.0 features including human in the loop with interrupt(), flow control with Command, long-term memory with Store, and langgraph-supervisor.
2. FastAPI Service: Serves the agent with both streaming and non-streaming endpoints.
3. Advanced Streaming: A novel approach to support both token-based and message-based streaming.
4. AG-UI Protocol Support: Every agent is also served over the AG-UI protocol for connecting AG-UI compatible frontends like CopilotKit - see docs.
5. Streamlit Interface: Provides a user-friendly chat interface for interacting with the agent, including voice input and output.
6. Multiple Agent Support: Run multiple agents in the service and call by URL path. Available agents and models are described in /info
7. Asynchronous Design: Utilizes async/await for efficient handling of concurrent requests.
8. Content Moderation: Implements Safeguard for content moderation (requires Groq API key).
9. RAG Agent: A basic RAG agent implementation using ChromaDB (optional rag extra) - see docs.
10. Chat History: Lists a user's previous conversations per agent via /threads, with a "Previous Chats" sidebar in the Streamlit app.
11. Feedback Mechanism: Includes a star-based feedback system integrated with LangSmith.
12. Docker Support: Includes Dockerfiles and a docker compose file for easy development and deployment.
13. Testing: Includes robust unit and integration tests for the full repo.

Key Files

The repository is structured as follows:

• src/agents/: Defines several agents with different capabilities
• src/schema/: Defines the protocol schema
• src/core/: Core modules including LLM definition and settings
• src/service/service.py: FastAPI service to serve the agents
• src/client/client.py: Client to interact with the agent service
• src/streamlit_app.py: Streamlit app providing a chat interface
• tests/: Unit and integration tests

Setup and Usage

1. Clone the repository:

   git clone https://github.com/JoshuaC215/agent-service-toolkit.git
   cd agent-service-toolkit
   

2. Set up environment variables: Create a .env file in the root directory. At least one LLM API key or configuration is required. See the .env.example file for a full list of available environment variables, including a variety of model provider API keys, header-based authentication, LangSmith tracing, testing and development modes, and OpenWeatherMap API key.

3. You can now run the agent service and the Streamlit app locally, either with Docker or just using Python. The Docker setup is recommended for simpler environment setup and immediate reloading of the services when you make changes to your code.

Additional setup for specific AI providers

• Setting up Ollama
• Running local models on local hardware (vLLM / SGLang)
• Setting up VertexAI
• Setting up RAG with ChromaDB

Building or customizing your own agent

To customize the agent for your own use case:

This is the beginning of the README. Read the rest at the original. Read the full README ↗