LLMai

---

What LLMai is

A privacy-first AI coding agent that runs the model locally and gives it three distinct kinds of awareness — not just one giant chat window:

Layer	Backed by	What it gives the agent
Operational	Dynatrace (OpenTelemetry → Bindplane)	Every tool call traced: latency, token count, permission outcome, success/error
Personal	MongoDB Atlas (per-workspace)	Recalls your past sessions, decisions, and extracted facts. Boots warm
Organizational	Elastic (per-org)	Hybrid search over GitLab issues, CI failure logs, docs — checks "have we seen this before?" before writing code

All three are opt-in. The default runtime stays local with Ollama. Cloud model, MCP, memory, search, and telemetry connections are enabled separately.

Live demo: ll-mai.vercel.app · Demo video (3 min): youtu.be/V3Rda1kLSos

---

Stack at a glance

                                        ┌─ search_knowledge ──► Elastic (issues + docs)
                                        │
   Browser / CLI ──► Agent Loop ────────┼─ recall_memory ─────► MongoDB Atlas
                          │             │                       (per-workspace sessions
                          │             │                        + summaries + knowledge)
                          │             │
                          │             └─ query_logs ────────► Elastic ES|QL
                          │                                     (pipeline + agent self-logs)
                          │
                          ▼
                     OTel spans
                          │
                          ▼
                  Bindplane ──┬──► Dynatrace (traces + metrics)
                              │
                              └──► Elastic llmai-agent-logs
                                   (so the agent can query itself)

LLM backend: Ollama by default; Gemini 3 and Google ADK are optional

---

Two interfaces

	How to run	Best for
CLI REPL	llmai	Terminal power users
Web UI	llmai-server → http://localhost:7777	Browser, streaming, permission cards

The server no longer pops a browser window on start; set LLMAI_OPEN_BROWSER=1 if you want the old behavior.

---

Daily briefing

The local server also serves a daily briefing dashboard at http://localhost:7777/briefing — and the hosted site's ☀️ Briefing tab embeds it directly (the server grants scoped CORS to /healthz and /briefing* for https://ll-mai.vercel.app only; override with LLMAI_BRIEFING_ORIGINS).

Each briefing contains:

• DS interview practice — one question + model answer, generated by your local Ollama model; topic rotates daily across 8 areas
• Korea news — KBS World / Korea JoongAng Daily RSS
• US market headlines — Yahoo Finance RSS with MarketWatch fallback

The briefing auto-regenerates in the background on server startup whenever the saved one isn't from today; POST /briefing/refresh forces a rebuild (also wired to the ↻ button in the tab).

---

Quick start (core only, no partner integrations)

Install from PyPI (the distribution is named llmai-agent because bare llmai was already taken; the import path is still import llmai):

pip install llmai-agent
ollama serve
ollama pull qwen2.5-coder
llmai-server   # → http://localhost:7777

Or install from source for development:

git clone https://github.com/sechan9999/LLMai.git
cd LLMai
pip install -e .

ollama serve
ollama pull qwen2.5-coder

llmai-server   # → http://localhost:7777

That's it. The agent has 8 core tools (read/write/edit files, run shell, search code, list files, fetch URL, mkdir) plus 11 GitLab tools when GITLAB_TOKEN is set. Read-only ops auto-approve; writes and shell commands prompt for permission.

---

Adding the three layers (10-15 min each)

1. Dynatrace — observe every tool call

make install-telemetry          # adds opentelemetry packages

# in .env (copy from .env.example):
DT_ENDPOINT=https://<your>.live.dynatrace.com
DT_API_TOKEN=dt0c01.YOUR_TOKEN

make bindplane-up               # starts the OTel collector locally

export LLMAI_OTEL_ENABLED=true
export LLMAI_OTEL_ENDPOINT=http://localhost:4318
llmai-server

Spans: agent.turn → agent.iteration → llm.chat + tool.invocation. Metrics: tool invocation counts, LLM latency, token histograms.

Full guide: docs/dynatrace-setup.md

2. MongoDB Atlas — remember across sessions

make install-memory             # adds pymongo
ollama pull nomic-embed-text    # embedding model

# in config.json or env:
LLMAI_MEMORY_ENABLED=true
LLMAI_MEMORY_URI=mongodb+srv://USER:[email protected]/

python scripts/setup_atlas_indexes.py   # one-time vector index bootstrap
llmai-server

Three collections: sessions (metadata-only by default), summaries (vector-embedded), knowledge (extracted facts, vector-embedded). New tool: recall_memory for the agent. On each new session, the 3 most recent prior summaries are auto-injected as a system message. Full transcript retention requires explicit LLMAI_MEMORY_STORE_TRANSCRIPTS=true.

Full guide: docs/atlas-setup.md

3. Elastic — search org knowledge before writing code

make install-elastic            # adds elasticsearch client
make elastic-up                 # local ES + Kibana via docker
make elastic-setup              # creates llmai-* indices

# optional: pull org knowledge in
export GITLAB_TOKEN=glpat-...
export GITLAB_PROJECT=group/project
make elastic-ingest             # pulls last 500 issues + 50 failed pipelines

export LLMAI_ELASTIC_ENABLED=true
export LLMAI_ELASTIC_URL=http://localhost:9200
llmai-server

Two tools: search_knowledge (hybrid keyword + dense vector, RRF-fused, auto-approved) and query_logs (raw ES|QL, permission-gated). System prompt nudges the model to call search_knowledge before writing code that touches error paths or external APIs.

Full guide: docs/elastic-setup.md

4. MCP — plug in partner MCP servers (GitLab, MongoDB, Elastic, …)

pip install 'llmai-agent[mcp]'   # MCP Python SDK (servers below also need Node >= 18)

# in config.json — GitLab's official MCP server via the mcp-remote bridge:
#   "mcp": { "enabled": true, "servers": { "gitlab": {
#       "command": "npx",
#       "args": ["-y", "mcp-remote", "https://gitlab.com/api/v4/mcp"],
#       "allow": [] } } }

llmai-doctor    # verify SDK + server commands
llmai-server    # tools appear as mcp__gitlab__*, mcp__mongodb__find, ...

Every MCP tool defaults to ask; only names in the server's allow list auto-approve. Servers run as local stdio subprocesses — no sockets.

Full guide: docs/mcp-setup.md

One-command demo bootstrap

make demo-up           # starts Elastic + Kibana + Bindplane
make demo-bootstrap    # pulls embed model + creates ES indices
make demo-status       # health check across the whole stack

---

Key features

• 100% local by default — no API keys, no cloud, nothing leaves your machine
• Agentic loop — observe → judge → act, up to 20 iterations per turn
• Permission gates — read-only auto-approves; writes and shell prompt
• Three-layer awareness — operational (Dynatrace), personal (Atlas), org (Elastic) — all opt-in
• Real MCP client — connect official partner MCP servers (GitLab, MongoDB, Elastic, or any stdio server); their tools register as mcp__{server}__{tool} behind the permission gate
• Dual tool-calling modes — native OpenAI function calling for capable models; XML fallback for gemma3, phi3, etc.
• Context compression — auto-summarizes when conversation exceeds ~50k tokens
• Workspace sandboxing — file ops restricted to WORKSPACE_ROOT; dangerous command patterns blocked
• Provider-agnostic — Ollama, Gemini, Groq (cloud fallback for the hosted demo)
• Dual interface — CLI REPL + WebSocket-streaming Web UI
• Daily briefing — DS interview practice + Korea news + US markets, generated locally and embedded in the site's Briefing tab

---

Tools (when their layer is enabled)

Category	Tools	Default permission
Core (8)	read_file, write_file, edit_file, run_command, list_files, search_code, fetch_url, create_directory	reads allow, writes/shell ask
GitLab (11)	gitlab_list_issues, gitlab_get_mr, gitlab_get_job_log, …	reads allow, mutations ask
Memory (1)	recall_memory	allow
Elastic (2)	search_knowledge, query_logs	allow / ask
MCP (dynamic)	mcp__{server}__{tool} — discovered from configured MCP servers	ask (per-server allow list)

Tools are registered conditionally — the model never sees a tool whose backend isn't connected.

---

Configuration

config.json (or env vars — env always wins):

{
  "ollama_url": "http://localhost:11434",
  "model": "qwen2.5-coder",
  "permissions": { ... },
  "telemetry": { "enabled": false, "endpoint": "http://localhost:4318", ... },
  "memory":    { "enabled": false, "uri": "mongodb+srv://...", ... },
  "elastic":   { "enabled": false, "url": "http://localhost:9200", ... }
}

Full example: config.example.json.

---

CLI commands

Command	Description
/reset	Clear conversation context (and finalize session memory)
/model <name>	Switch model
/models	List Ollama models locally
/tokens	Show estimated token count
/perms	Show current permission settings
/compress	Force context compression now
/exit	Quit

---

Supported models

Native tool calling (recommended): Qwen 2.5 / 2.5-Coder / 3, Llama 3.1+, Mistral NeMo, FireFunction, Command-R(+).

XML fallback: gemma3, phi3, mistral — anything OpenAI-incompatible. Slightly lower fidelity but functional.

---

Cloud backends (optional)

The agent speaks the OpenAI /v1/chat/completions contract.

# Gemini 3 through the existing CLI and Web UI agent loop
export GEMINI_API_KEY=...
export GEMINI_MODEL=gemini-3.1-pro-preview   # optional — this is already the default (llmai/llm.py); set only to override, e.g. gemini-3.5-flash

llmai-doctor   # validates the key and model availability

# Any OpenAI-compat endpoint (LM Studio, vLLM, custom):
# set "ollama_url" in config.json

Google Cloud Agent Builder profile

LLMai also includes an optional Google Agent Development Kit (ADK) entry point. In this profile, a Gemini 3 model orchestrates read-only workspace tools and GitLab's official MCP server. It is separate from the default Web UI runtime and does not claim to reproduce the Web UI's interactive write approvals.

pip install -e ".[google-cloud]"
export GOOGLE_API_KEY=...
export LLMAI_GOOGLE_MODEL=gemini-3.1-pro-preview
adk web .

The discovered agent is google_agent. Its local and GitLab tools are filtered to read-only operations. See docs/google-cloud-agent-builder.md for AI Studio, Vertex AI, OAuth, and verification steps.

---

Project structure

LLMai/
├── llmai/                # Python package (formerly vixcode)
│   ├── agent.py          # Sync CLI agent loop
│   ├── main.py           # CLI REPL entry point
│   ├── llm.py            # OpenAI-compat HTTP client
│   ├── tools.py          # 8 core tools + conditional registration
│   ├── gitlab_tools.py   # 11 GitLab tools
│   ├── permissions.py    # allow / ask / deny system
│   ├── telemetry.py      # OpenTelemetry init + span context managers
│   ├── memory/           # MongoDB Atlas persistent memory
│   │   ├── store.py
│   │   ├── embeddings.py
│   │   └── recall_tool.py
│   ├── elastic/          # Elasticsearch knowledge search + log analytics
│   │   ├── client.py
│   │   ├── search_tool.py
│   │   └── query_tool.py
│   └── mcp/              # MCP client: partner servers over stdio
│       ├── client.py
│       ├── bridge.py
│       └── registry.py
├── google_agent/         # Optional Gemini 3 + Google ADK entry point
├── server/               # FastAPI + WebSocket Web UI
│   ├── app.py            # Routes + daily briefing (auto-regen, scoped CORS)
│   ├── agent_ws.py       # Async agent loop (native + XML modes)
│   └── static/           # Web UI + briefing dashboard/template
├── website/              # Hosted site (Vercel)
│   ├── index.html        # Chat + Briefing tab + About + Install
│   ├── landing.html      # Marketing landing page
│   └── promo.html        # 60-second animated demo
├── api/chat.js           # Vercel serverless cloud fallback (Groq)
├── scripts/              # Bootstrap scripts for partner integrations
│   ├── setup_atlas_indexes.py
│   ├── elastic_setup_indexes.py
│   ├── elastic_ingest_gitlab.py
│   └── elastic_ingest_logs.py
├── bindplane/config.yaml             # OTel collector → Dynatrace + Elastic
├── docker-compose.bindplane.yml      # Bindplane container
├── docker-compose.elastic.yml        # Elasticsearch + Kibana
├── Makefile                          # Common dev / demo tasks
├── docs/
│   ├── dynatrace-setup.md
│   ├── atlas-setup.md
│   └── elastic-setup.md
└── tests/                # 103 passing

---

Documentation

• Dynatrace observability — OpenTelemetry spans, metrics, Bindplane pipeline
• MongoDB Atlas memory — cross-session continuity, semantic recall
• Elastic knowledge search — hybrid search, ES|QL, agent self-logs
• MCP integration — partner MCP servers (GitLab, MongoDB, Elastic) over stdio
• Google Cloud Agent Builder — Gemini 3 + ADK + GitLab MCP verification path
• Local LLM Setup — best practices for Ollama
• Permission System — allow/ask/deny configuration
• Tool System — tool definitions and sandboxing
• OSS Comparison — vs. Claude Code, Aider, Open Interpreter, Goose

---

License

MIT

---

Acknowledgments

• Ollama — Local LLM runtime
• FastAPI — Web framework
• Rich — Terminal formatting
• OpenTelemetry — Observability standard
• Bindplane — OTel collector
• Dynatrace, MongoDB Atlas, Elastic — Hackathon partner backends