Overview
Roshi is a Postman-like desktop client built specifically for LLM APIs. It understands multi-turn messages, streaming, model parameters, token usage, provider-specific payloads, and model comparisons.
Use Roshi with OpenAI, Anthropic, Google Gemini, OpenRouter, or any OpenAI-compatible endpoint. Build a request once, inspect exactly what was sent and returned, then compare the same prompt across models by quality, latency, token usage, and cost.
| Area | Details |
|---|
| Providers | OpenAI, Anthropic, Google Gemini, OpenRouter, and OpenAI-compatible APIs |
| Core workflow | Compose → send → inspect → compare → evaluate |
| Data storage | Local IndexedDB; no Roshi backend, account, telemetry, or hosted key storage |
| Desktop platforms | macOS on Apple Silicon and Intel |
| Model evaluation | Side-by-side runs, manual ratings, LLM-as-judge scoring, response diffs, and CSV/JSON export |
Workflow
- Add a provider. Choose a built-in template or configure a compatible endpoint.
- Compose a request. Add messages, images, model parameters, and custom headers.
- Inspect the result. Review streamed text, raw payloads, timing, and token usage.
- Compare and evaluate. Run the request across models, apply your rubric, and save or export the results.
Screenshots
These are current views from the shipping app.
Install
Roshi supports macOS on Apple Silicon (M1+) and Intel.
To install a GitHub release:
- Download the
.dmg for your Mac from the latest release.
- Open it and drag Roshi to Applications.
- If macOS blocks the first launch, open System Settings → Privacy & Security and select Open Anyway.
- Add a provider API key and start testing.
The browser build is intended for development. It only works with CORS-enabled providers; use the desktop app for full provider compatibility.
Features
Build and debug requests
- Multi-turn conversations with system prompts, image attachments, sampling parameters, and custom headers
- Real-time SSE streaming with raw request and response inspection
- Token usage, latency, time-to-first-token, throughput, and estimated cost metrics
- Python and Node.js snippets for supported OpenAI-compatible requests
Organize your work
- Model catalog powered by models.dev
- Up to eight independent request tabs
- Searchable local history with filtering and response diffs
- Reusable collections and saved requests
- Named environments with
{{var}} substitution
⌘K command palette and light/dark themes
Compare models
- Side-by-side runs across multiple providers and models
- Manual ratings and custom LLM-as-judge rubrics
- Word-level response diffs and direct metric comparisons
- Saved eval runs with CSV and JSON export
Privacy and security
Roshi is open source under the MIT License. It has no backend, account system, telemetry, or hosted dependency. The desktop app calls provider APIs directly, and CI checks formatting, linting, tests, and production builds.
API keys are stored unencrypted in local IndexedDB:
- Desktop (recommended for real keys): keys are sent directly to the selected provider, never through Roshi infrastructure. The Tauri app uses a restrictive Content Security Policy.
- Web or self-hosted: an XSS vulnerability on the hosting origin could read stored keys. Use the browser build for development and testing, not for production secrets.
Development
Looking to use Roshi? Install the desktop app. This section is for contributors.
Prerequisites
- Node.js 26+
- Rust (for Tauri desktop builds)
Quick start
npm install
npm run dev
Then open http://localhost:5173.
The dev server includes a CORS proxy at /api/proxy. Production web builds do not, so providers that block browser requests will fail there. Use the Tauri desktop app for full provider compatibility.
Commands
npm run build # Production build
npm run test # Unit and integration tests
npm run test:e2e # Playwright end-to-end tests
npm run lint # ESLint
npm run typecheck # TypeScript
npm run format # Prettier
npm run tauri:dev # Desktop app in development
npm run tauri:build # Desktop release build
Tech stack
React 19 · TypeScript · Vite 7 · Tailwind CSS v4 · shadcn/ui v4 · Zustand · Dexie.js · Tauri 2 · Vitest · Playwright
See AGENTS.md for architecture, conventions, and contributor documentation.
Contributing
Issues and pull requests are welcome. To suggest a feature or report a bug, open an issue with enough detail to reproduce the behavior.
Author
Created by Andrew Mok (@andrewmmc)
Disclaimer
Roshi is not affiliated with, endorsed by, or sponsored by Postman, Inc. “Postman” is a trademark of Postman, Inc. References to Postman are descriptive only.
License
MIT License.
