Traccia SDK for Javascript/TypeScript

Production-ready distributed tracing for AI agents and LLM applications

Traccia is a lightweight, high-performance Javascript/TypeScript SDK for observability, evaluation, and tracing of AI agents, LLM applications, and complex distributed systems. Built on OpenTelemetry standards with specialized instrumentation for AI workloads.

---

Features

• Automatic Instrumentation: Auto-patch OpenAI, Anthropic, Gemini (@google/genai), LangChain support.
• LLM-Aware Tracing: Track tokens, costs, prompts, and completions automatically.
• Zero-Config Start: Simple startTracing() call with automatic config discovery.
• Decorator-Based: Trace any function with the @observe decorator.
• Multiple Exporters: OTLP (compatible with Grafana Tempo, Jaeger, Zipkin) and Console exporters.
• Production-Ready: Rate limiting, typed errors, config validation, robust flushing.
• Type-Safe: Full TypeScript support with TracciaError hierarchy.
• High Performance: Efficient batching, async support, minimal overhead.
• W3C Trace Context: Native distributed tracing header propagation.
• Governance & Policies: Trace evidence, disclosure(), and govern() which can deny or reshape this LLM or tool call (Spend Cap, Model Boundary, Loop Cap) against the Traccia platform. Refund Guard and Purchase Guard can also hold a tool for a person (ApprovalPending).
• Agent Identity: Centralized configuration mapping to OTel resource attributes.
• Prompt Management: loadPrompt / prefetchPrompts with cache, stale-while-revalidate, fallback, and traccia.prompt.* span identity.
• Offline Evaluation: evaluate() runs a task and scorers over a dataset and saves an experiment you can open, compare, and attach on promote.

---

Quick Start

Installation

npm install @traccia/sdk

Basic Usage

Initialize the SDK and trace your application functions using decorators.

import { Traccia, observe } from '@traccia/sdk';

// Initialize (auto-loads from traccia.toml or env vars if present)
await Traccia.init();

// Trace any function
class Service {
  @observe()
  async doWork(input: string) {
    return `Processed: ${input}`;
  }
}

const service = new Service();
await service.doWork('hello');

Load a versioned prompt

Fetch prompts from your Traccia library by name and deploy label. Compiling fills {{variables}} and stamps the active span with prompt identity.

import { init, observe, loadPrompt, prefetchPrompts } from '@traccia/sdk';

await init({ apiKey: '...', promptCacheTtlS: 60 });
await prefetchPrompts(['support-reply']); // optional warm-up

const reply = observe({ type: 'llm' })(async (question: string) => {
  const prompt = await loadPrompt({
    name: 'support-reply',
    label: 'production',
    fallback: {
      type: 'chat',
      messages: [{ role: 'system', content: 'You are a helpful assistant.' }],
    },
  });
  const messages = prompt.compile({ question });
  // pass messages to your LLM client
  return messages;
});

Requires the Traccia platform: create a workspace API key under Settings → API Keys. Tracing and prompt fetch use the same key and default to https://api.traccia.ai. Pass an explicit fallback so agents can still run if a fetch fails. See Prompts in the SDK.

Run an experiment

evaluate() runs a task and scorers over a dataset. By default the run is saved as an experiment you can open in the app under Evaluate → Experiments, compare against other runs, and attach when you promote a prompt. Use the same workspace API key as tracing and loadPrompt.

import { init, evaluate, loadPrompt } from "@traccia/sdk";

await init({ apiKey: "..." });

const prompt = await loadPrompt({ name: "support-reply", label: "production" });

const result = await evaluate({
  name: "support-reply-v3",
  data: "support-golden", // platform dataset name or UUID
  task: async (inp) => {
    const messages = prompt.compile(inp);
    return callModel(messages);
  },
  scorers: ["exact_match"], // builtins, platform names/ids, or functions
  prompt: "support-reply",  // labels the experiment cell with this prompt
});
console.log(result.summary());
console.log(result.url);

• Persist is on by default. Set persist: false for a local-only loop (no experiment URL).
• Inline rows work as [{ input: { ... }, expected: "..." }]. With persist on, Traccia still saves a full experiment (helper datasets named sdk-eval/... are hidden on Evaluate → Datasets unless you turn on Show SDK-Created).
• Builtins run in-process: exact_match, contains, json_valid. Mix in platform scorers (LLM-as-judge, Jev Decision, code) by name or UUID. Pass providerKeys for judges (typesafe for Jev). A Jev score keeps each question on the experiment.
• One throwing row does not abort the run. That cell records error; other items still score. Configuration and API failures raise EvaluateError.
• The task always receives the row's input object only. Wrapper fields are camelCase (experimentId, persistError); row payloads stay snake_case.

See Evaluate in the SDK and the walkthrough Run Experiments From Code.

Run evals in CI

traccia-ts eval runs a module that exports task, saves an experiment, and writes summary.json. Exit codes match the Python CLI: 0 pass, 1 quality miss, 2 bad command or baseline, 3 untrustworthy evidence, 4 the experiment did not save, 5 the summary file was not written. With no pass-rate bar, a green check is Execution Only: no pass-rate gate, while cost and latency gates can still apply. --max-drop 0.05 is five percentage points. A function in the module is export const scorers = { name } and --scorer local:name. A local scorer cannot pass --max-drop. Pass --prompt with --prompt-version or --prompt-label when the experiment should record a prompt version. Platform LLM-judge and Jev scorers read OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY or GOOGLE_API_KEY, GROQ_API_KEY, and TYPESAFE_API_KEY, or a JSON object in TRACCIA_PROVIDER_KEYS. Each Jev question is gated on its own pass rate. Load .ts task files with tsx.

traccia-ts eval experiments/support_agent.mjs \
  --output summary.json \
  --dataset support-golden \
  --scorer exact_match \
  --min-pass-rate 0.9

See Run Evals In CI.

Integration with LLMs (LangChain)

Seamlessly integrate with LangChain to automatically track model usage, tokens, and latency.

import { Traccia, observe } from '@traccia/sdk';
import { ChatOpenAI } from '@langchain/openai';
import { TracciaCallbackHandler } from '@traccia/sdk/integrations';

await Traccia.init();

const model = new ChatOpenAI({
  modelName: 'gpt-4',
  callbacks: [new TracciaCallbackHandler()],
});

@observe({ type: 'llm' })
async function generateText(prompt: string) {
  const res = await model.invoke(prompt);
  return res.content;
}

// Automatically tracks: model, tokens, cost, prompt, completion, latency
const text = await generateText("Write a haiku about TypeScript");

Gemini (@google/genai)

Wrap client.interactions.create to trace the Gemini Interactions API (@google/genai >=2.9.0, tested against 2.17.1). This is the only Gemini surface instrumented today — models.generateContent is not covered.

import { Traccia, wrapGeminiInteractionsCreate } from '@traccia/sdk';
import { GoogleGenAI } from '@google/genai';

await Traccia.init();

const client = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

client.interactions.create = wrapGeminiInteractionsCreate(
  client.interactions.create.bind(client.interactions),
  client.interactions,
);

const response = await client.interactions.create({
  model: 'models/gemini-2.5-flash',
  input: 'Write a haiku about TypeScript',
});

Captures llm.model, llm.prompt, llm.completion, llm.previous_interaction_id (multi-turn), and token usage read directly from the provider's usage.total_* fields (total_input_tokens, total_output_tokens, total_thought_tokens, total_cached_tokens, total_tool_use_tokens, total_tokens) — never synthesized from input+output, so thinking/cache/tool tokens aren't dropped.

Streaming (stream: true): the span is still created and ends with request attributes (llm.streaming: true), but usage/completion population is intentionally skipped — create() resolves with a Stream object almost immediately, before the model has produced output, so populating from it would record a near-zero duration and no usage data.

---

Configuration

Configuration File

Create a traccia.toml file in your project root using the CLI:

npx traccia-ts config init

This creates a template config file with the following structure:

[tracing]
# Endpoint URL for OTLP trace ingestion (Traccia platform default)
endpoint = "https://api.traccia.ai/v2/traces"

# Flush interval for batching spans (in ms)
flush_interval = 5000

# Maximum size of span batch before forced flush
batch_size = 512

[agent]
# Name of the agent emitting traces
name = "my-typescript-agent"

# Type of the agent (e.g., 'workflow', 'llm', 'retriever')
type = "workflow"

# Environment (e.g., 'development', 'staging', 'production')
env = "development"

# Project this agent belongs to
project = "my-default-project"

[integrations]
# Enable/disable specific integrations
langchain = true
openai = true
gemini = true

Environment Variables

Configuration can also be provided via standard environment variables:

• TRACCIA_API_KEY: API Key for ingestion authentication.
• TRACCIA_ENDPOINT: The OTLP endpoint (default: https://api.traccia.ai/v2/traces).
• TRACCIA_AGENT_NAME: The name of the agent tracing execution.
• TRACCIA_ENV: The environment (e.g., development, production).

---

Advanced Usage

Custom Traces and Attributes

You can manually trace components and add custom business metrics to spans:

import { trace } from '@opentelemetry/api';
import { observe } from '@traccia/sdk';

class DataPipeline {
  @observe({ type: 'retriever', name: 'fetch_user_data' })
  async fetchData(userId: string) {
    const activeSpan = trace.getActiveSpan();
    
    // Add custom business metrics
    activeSpan?.setAttribute('app.user.id', userId);
    activeSpan?.setAttribute('db.query.latency', 45);
    
    return { id: userId, status: 'active' };
  }
}

Stopping the SDK

To ensure all queued telemetry data is successfully flushed to the backend before the process exits, invoke stopTracing():

import { stopTracing } from '@traccia/sdk';

// Flush buffered spans and shut down the provider safely
await stopTracing();

evaluate() options

evaluate() is async and takes one options object. Also available as Traccia.evaluate.

Option	Notes
name	Experiment name (required)
data	Dataset name/UUID, or inline rows { input, expected, metadata } (expected_output aliases expected)
task	(input) => output. Always receives input only. Async is awaited
scorers	Builtin ids exact_match, contains, json_valid; platform names/ids; and/or functions
prompt	Optional prompt name for version ids and cell label (otherwise Task)
maxConcurrency	Parallel workers (default 10)
persist	Create an experiment (default true)
providerKeys	BYO keys for platform LLM-as-judge, and typesafe for Jev Decision. A Jev score keeps each question, so the experiment shows one chip per question.
progress	Print N/M to stderr (default true)

Returns EvaluateResult with rows, aggregates, summary(), url, experimentId, datasetId, errors, persistError. Empty data, bad config, and dataset/scorer API failures throw EvaluateError. Full reference: Evaluate in the SDK.

---

Governance

observe() vs govern()

API	Purpose	Requires Traccia platform
observe()	Observability only	No
govern()	Observability + runtime policy enforcement	Yes

govern() does two things:

1. Checks agent status before the function body (can warn or stop the next run for after-ingest policies).
2. Turns on a per-call check for this LLM or tool call (Spend Cap, Model Boundary, Loop Cap). You do not call checkPolicy() yourself when using an instrumented LLM client and observe({ asType: 'tool' }).

Tracing-only or self-hosted users should use observe().

import { OpenAI } from 'openai';
import { init, observe, govern, AgentBlockedError } from '@traccia/sdk';

await init({ apiKey: '...', endpoint: 'https://api.traccia.ai/v2/traces', agentId: 'my-agent' });
const client = new OpenAI();

const lookupOrder = observe({ name: 'lookup_order', asType: 'tool' })(
  async (orderId: string) => ({ order_id: orderId, status: 'shipped' }),
);

const runAgent = govern({
  name: 'run_agent',
})(async (prompt: string) => {
  await lookupOrder('ORD-1');
  const resp = await client.chat.completions.create({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: prompt }],
  });
  return resp.choices[0]?.message?.content ?? '';
});

try {
  await runAgent('hello');
} catch (err) {
  if (err instanceof AgentBlockedError) {
    console.log(err.reasons, err.decisionId, err.remainingBudgetUsd);
  }
}

Identity comes from init({ agentId }) (or TRACCIA_AGENT_ID). Pass agentId on govern() only to override in a multi-agent process. Without an agent ID, API key or endpoint, govern() throws GovernanceConfigError (a subclass of AgentBlockedError) instead of running unchecked. On Block deny, govern() throws AgentBlockedError. Observe and Warn still let the call proceed and record a match. Refund Guard and Purchase Guard can hold a middle amount: govern() throws ApprovalPending, which is not AgentBlockedError. Catch it, do not run the tool, and do not retry the check. pendingToolResult() is a normal tool result when a framework retries raised errors.

observe({ asType: 'tool' }) matches Python @observe(as_type="tool"). type: 'tool' still works.

For a custom tool that is not wrapped with observe({ asType: 'tool' }), call checkPolicy({ action: { type: 'tool_call', name: 'refund' }, context: { input: { amount } } }) yourself.

When Traccia can't be reached

If a check gets no answer (timeout, connection error, 5xx, or an invalid answer), the call follows your policy fail mode: Settings → Policy Enforcement in the app, for the organization or one workspace. The default is to block: the call throws AgentBlockedError, and the provider or tool is never called. Choose Allow there to let calls through while Traccia is down.

• The setting arrives with answers the SDK already gets: the agent status at the start of each govern() run (cached 60 s) and each policy check. There are no extra requests, and tracing-only code never calls Traccia for it.
• It is kept for 10 minutes after the last answer that carried it, then the SDK goes back to blocking. A process that has had no answer yet, for example one that starts while Traccia is down, blocks. Calling init() again forgets it.
• A local override applies only when there is no answer: init({ policyFailMode: 'open' | 'closed' }) for the process, or govern({ policyFailMode }) for one run and the checks inside it. It never overrides a rejected API key (401/403), a block, a deny or a configuration error. The SDK logs a warning once when it disagrees with your Traccia setting, since the Traccia Gateway follows the setting.
• govern({ failOpen }) and checkAgentStatus(id, { failOpen }) are deprecated: true means policyFailMode: 'open', false means 'closed'. Omit it to follow your setting.

Policies in the app: Policies. SDK guide: Governance in the SDK.

Advanced (optional): override endpoints in traccia.toml:

[governance]
status_check_endpoint = "https://custom.example/agents/{agent_id}/status"
post_block_endpoint = "https://custom.example/agents/{agent_id}/blocks"
status_cache_ttl_seconds = 120
policy_fail_mode = "closed"   # optional local override, see above

Transparency — disclosure()

Record EU AI Act Art. 50 transparency evidence on the active span when your UI shows an AI disclosure:

import { disclosure } from '@traccia/sdk';

disclosure({ channel: 'ui', disclosedToUser: true });

HIPAA-oriented init

init({
  compliance: { frameworks: ['hipaa'] },
  redactPii: true, // recommended; defaults on when hipaa is in frameworks unless set false
});

Soft warnings only — Traccia does not block PHI. See HIPAA docs and the Trust Center. No signed BAA yet — contact [email protected].

---

License

This project is licensed under the MIT License. See the LICENSE file for details.