🏆 First-Place Winner — Best use of Google Gemini (MLH) at HackGT 2025
Ming is an AI-native, multi-agent L&D platform that transform current and aspiring engineer to managers via practicing real-world workplace scenarios (performance reviews, conflict resolution, even layoffs) through LLM-driven role-play with detailed analysis and actionable feedback.
Why Ming?
- Bridge technical → managerial: Upskill engineers into empathetic, effective managers—faster and more affordably than hiring externally.
- Soft-skills that scale: High-quality practice for communication, coaching, and conflict resolution—skills traditional trainings rarely teach well.
- AI-economy ready: As automation reshapes work, Ming helps people build durable, human-centered competencies.
What it does
- Role-played conversations with LLM NPCs across tricky scenarios (reviews, conflicts, reorganizations, exits).
- Real-time voice & signals: Live sentiment analysis, pacing, interruptions, and conversational grounding.
- Deep debriefs: Annotated transcripts, strengths/areas to improve, and personalized follow-ups.
- Adaptive curriculum: New scenarios are generated from each user’s performance profile.
Interface
How we built it
- Frontend: Next.js (App Router) + React, Tailwind, shadcn/ui
- Orchestration: CedarOS to integrate frontend with multi-agent runtime
- Current backend: Mastra with Gemini text and analysis by default (OpenAI selectable), OpenAI Live voice by default, and optional Deepgram speech-to-text/text-to-speech.
- Local demo: Next.js on localhost with a private Node/Mastra backend. The app opens directly without a login. See local run details.
- Architecture: Multi-agent LLMs coordinate roles (manager, employee, facilitator/coach) for realistic dynamics
Quickstart
Prereqs: Node 24 LTS and npm
npm ci
npm ci --prefix src/backend
cp .env.example .env
# Add GEMINI_API_KEY and OPENAI_API_KEY for the default providers.
npm run dev
Open http://127.0.0.1:3000 (or http://localhost:3000). There is no login,
access password, or session-secret setup. npm run dev loads the root .env,
creates a temporary internal gateway token, and starts both services with the
same token. It does not change .env or print the token. Stop with Ctrl+C;
restart after changing .env.
Provider selection is independent:
| Setting | Default | Alternative |
|---|
MING_TEXT_PROVIDER | gemini — GEMINI_API_KEY | openai — OPENAI_API_KEY |
MING_VOICE_PROVIDER | openai — OPENAI_API_KEY | deepgram — DEEPGRAM_API_KEY |
The defaults need both Gemini and OpenAI keys. To use Gemini + Deepgram, set
MING_VOICE_PROVIDER=deepgram and configure Gemini and Deepgram keys; an OpenAI
key is not needed. Restart npm run dev after changing providers. Missing keys
produce recoverable errors; the app never silently switches vendors.
GEMINI_MODEL and OPENAI_TEXT_MODEL override the text model. Gemini also accepts
GOOGLE_GENERATIVE_AI_API_KEY, then GOOGLE_API_KEY, when preceding keys are
unset or blank. The Veo helpers use the same server-side key precedence. The optional
MING_BACKEND_URL defaults to http://127.0.0.1:4111.
Leave MING_APP_ORIGIN unset for normal localhost use. A fixed
MING_BACKEND_TOKEN is only needed when running the services separately or
using the optional container setup.
The browser calls /api/backend; provider keys and the internal gateway token
stay on the server. NEXT_PUBLIC_URL is no longer used. This demo uses one
shared local identity and local conversation data. See run details and remaining
dependency advisories.
All backend text agents use the shared model in src/backend/src/mastra/models.ts.
Debriefs generate the summary and transcript annotations concurrently. The report
opens when its summary and scores are ready; transcript notes finish afterward
and can be retried independently. Annotations reference the original conversation
instead of asking the model to rewrite it. Successful stages and in-flight work
are reused for the same owner, session, transcript, context, provider, model, and
prompt version. Each stage cache holds at most 64 entries with a 15-minute TTL;
reuse is process-local and clears on backend restart. Bump REPORT_PROMPT_VERSION
when changing either report prompt or its generation settings. Backend
debrief_generation logs contain stage duration, success, and token counts, without
conversation text.
By default, voice conversations use OpenAI GPT-Live (gpt-live-1) over WebRTC. The root
OPENAI_API_KEY must belong to a project with GPT-Live access. Run on localhost
or HTTPS and allow microphone access when the chat opens. The voice connection
starts automatically with the microphone muted. Hold Space (outside the text
editor) or hold the microphone button to stream speech immediately; release only
mutes the microphone. Sending does not wait for release. The connection stays
open between presses. Use the phone button to end it.
End Scenario finishes voice transcription before generating the report.
The browser receives no OpenAI API key. Mastra supplies scenario facts and saves
the spoken transcript; typed chat, analysis, reports and delegated reasoning use
the selected text model.
With MING_VOICE_PROVIDER=deepgram, hold the microphone or Space to record one
utterance, then release to submit it. Deepgram transcribes the recording; the
selected text model generates Bart's reply; Deepgram speaks it. Unlike the Live
path, this mode waits for release before processing. It caps a recording at 30
seconds and preserves transcript text when speech playback fails. End Scenario
waits for the pending turn and saved transcript before generating feedback.
DEEPGRAM_STT_MODEL defaults to nova-3; DEEPGRAM_TTS_MODEL defaults to
aura-2-orion-en. Audio and provider keys go through the server.
Set MING_STT_PROVIDER=openai to transcribe your turns with OpenAI instead
(OPENAI_API_KEY, OPENAI_STT_MODEL, default gpt-transcribe). If OpenAI fails
or times out, Deepgram transcribes that same turn; Firefox's ogg recordings always
go to Deepgram, which stays required.
Set MING_TTS_PROVIDER=gemini to speak Bart's replies with Gemini text-to-speech
instead (any other value keeps Deepgram, the default). It uses the same Gemini
key as text, GEMINI_TTS_MODEL (default gemini-3.8-flash-lite-tts) and
GEMINI_TTS_VOICE (default Puck), and streams raw 24 kHz PCM to the browser.
If Gemini fails before any audio (quota, network, no key, empty output), that
same reply is spoken by Deepgram instead; Deepgram STT and its key stay required.
MING_TTS_PROVIDER=openai does the same with OpenAI text-to-speech, using
OPENAI_API_KEY, OPENAI_TTS_MODEL (default gpt-4o-mini-tts) and
OPENAI_TTS_VOICE (default cedar, the voice Bart uses in OpenAI Live mode).
OpenAI shuts down this endpoint on 2027-01-06; after that, Deepgram answers.
The Google Veo video-generation helpers remain in src/lib/videoUtil.ts and
src/backend/src/lib/videoUtil.ts. Their existing scenario-to-video behavior is
preserved; they now read the Gemini key server-side rather than a NEXT_PUBLIC_
key. They are not invoked automatically during the conversation demo.
Run npm run typecheck, npm run typecheck --prefix src/backend, npm run lint,
npm run test:voice, npm run test:providers, npm run test:deepgram,
npm run test:demo, and the security tests in the CI workflow.
Builds enforce types and lint. Runtime smoke checks are documented in
the local run guide.
Features (highlights)
- 🎭 Multi-agent scenarios with realistic personalities and constraints
- 🗣️ Realtime voice (speak to the NPCs; they speak back)
- 📈 Live sentiment & signals to nudge pacing and tone
- 🧠 Strengths/weaknesses modeling → personalized next scenarios
- 📝 Annotated transcripts with feedback and targeted micro-lessons
- ⚙️ Pluggable scenarios (JSON/YAML) for custom org content
Challenges we solved
- Cloudflare production split: We initially bundled Next.js and the Mastra server into a single Wrangler target, then split into two services and pointed the Next.js server to Mastra—stabilizing deploys and edge runtime behavior.
Inspiration
We want to help people learn and build skills that endure. Technical orgs need more managers who combine deep engineering instincts with excellent people leadership. Traditional trainings are generic and forgettable; Ming aims to be hands-on, adaptive, and effective.
Tech stack
- UI: Next.js, React, Tailwind, shadcn/ui
- AI/Agents: Mastra, Google Gemini (default text), OpenAI Live (default voice), and optional OpenAI text / Deepgram STT and TTS
- Orchestration: CedarOS
- Runtime: Local Next.js and Node/Mastra; the original HackGT version used Cloudflare
Roadmap
- 🧪 Stakeholder pilots & usability testing
- 🕊️ Multi-party conversations (e.g., mediating conflict between two coworkers)
- ⚡ Ultra-low-latency agent interactions
- 🥽 Immersive AR/VR practice environments
- 📱 Seamless mobile support
- 🔎 Culture-aware scaling: learn patterns within a company to tailor coaching
Acknowledgements
- Major League Hacking (MLH) — Best use of Google Gemini Prize (HackGT 2025) 🏅
- Thanks to mentors, judges, and early testers for feedback!
