MultimediaToolKit

Convert anything into anything: 5 formats (Text, Image, Audio, Video, Animation) and 20 directed conversions. Each one does a real operation and produces a real downloadable file.

• Frontend: Next.js 14 (App Router), TypeScript, Tailwind, dark and light mode, responsive
• Backend: FastAPI, SQLAlchemy 2, Alembic, PostgreSQL (SQLite for local dev), a durable DB-backed job queue (optional Redis wake-ups), and S3-compatible storage
• Engines: FFmpeg, faster-whisper, Kokoro-82M TTS, Tesseract OCR and Pillow, all on CPU. An optional hosted AI provider (Cloudflare Workers AI or Hugging Face) handles generative image models and image description.

Live demo: https://multimedia-tool-kit.vercel.app (free account: sign up with email and password) · API: https://mtk-backend.onrender.com/health · Presenting? See DEMO_GUIDE.md

---

The 20 conversions

Each conversion has at least one CPU mode that always works. AI modes need an AI provider to be configured (see AI provider). Without one they say so and are never faked.

#	Conversion	CPU modes	AI modes (provider)
1	Text → Audio	Kokoro neural TTS (9 voices, MP3/WAV/OGG/FLAC/M4A)	n/a
2	Audio → Text	Whisper transcript as TXT/SRT/VTT/JSON	n/a
3	Video → Text	FFmpeg + Whisper	n/a
4	Video → Audio	FFmpeg extraction (5 formats)	n/a
5	Animation → Text	On-screen text (Tesseract) + speech (Whisper)	Describe visuals
6	Animation → Audio	Extract soundtrack · read on-screen text aloud	Narrate visuals
7	Text → Image	Typography poster	AI image (FLUX.1-schnell)
8	Text → Video	Narrated scene video (cards + Kokoro + Ken Burns + captions)	AI-illustrated scenes
9	Text → Animation	Kinetic typography GIF/WebP	AI image loop
10	Audio → Image	Waveform art · spectrogram	AI cover art (speech + mood → prompt)
11	Audio → Video	Waves / spectrum / bars visualiser + Whisper captions	Visualiser on AI art
12	Audio → Animation	Waveform loop · beat pulse	AI art pulse
13	Image → Text	OCR (Tesseract)	Describe image
14	Image → Audio	Read text aloud (OCR → TTS)	Describe in speech
15	Image → Video	Ken Burns camera motion	AI-narrated video
16	Image → Animation	Seamless motion loop	n/a
17	Video → Image	Frame picker + edits · best frame · poster	AI poster
18	Video → Animation	Palette GIF/WebP, effects (cartoon, vintage, B&W), boomerang	n/a
19	Animation → Image	Best frame · poster · contact sheet	n/a
20	Animation → Video	MP4/WebM, loops, motion smoothing	n/a

The live status of every conversion and mode on a given server is available at GET /api/v1/conversions. If an engine is missing (for example Tesseract), the UI shows "Engine not installed" and the API refuses the job with 503.

Features

• Conversion workspace (/convert/[id]), generated from the backend registry: drag-and-drop upload, local preview (scrub a video to pick a frame), mode and settings forms, live progress with step messages, and audio, image and text previews with downloads.
• Conversion history (/jobs) with filters, errors and all output files. Projects and media library (/projects) with search, type filter, previews and delete.
• Public accounts: anyone can sign up with an email and password (scrypt-hashed). Sign-in issues a random, expiring session token, stored hashed. Sign-up and sign-in are rate-limited. Every user sees only their own files, jobs and history, and downloads use short-lived signed links.
• Reliability: durable job queue with leases, heartbeats and automatic recovery; FFmpeg timeouts; honest error codes (NO_SPEECH, NO_AUDIO, INPUT_TOO_LONG, AI_PROVIDER_ERROR, …); failed jobs leave no partial files behind.
• Security: server-derived MIME types, nosniff and sandbox CSP on downloads, upload allow-list and size cap, path-traversal-safe storage keys, and refusal to start in production with an unsafe configuration.

Architecture

Next.js (Vercel)  ──HTTPS + Bearer key──►  FastAPI API ──► PostgreSQL (jobs = durable queue, Alembic)
                                              │  ▲                 ▲
                                              ▼  │ signed URLs     │ claim / heartbeat / recover
                                     S3-compatible storage   Worker (embedded thread or `python -m app.worker`)
                                     (local disk in dev)        └─ registry → pipelines → adapters:
                                                                   FFmpeg · Whisper · Kokoro · Tesseract ·
                                                                   Pillow graphics/motion · AI provider (opt.)

Code map: backend/app/engines/registry.py (the 20 conversions), backend/app/engines/pipelines/ (workflows), backend/app/adapters/ (engines), frontend/src/app/convert/ (workspace UI).

Local development

Backend

cd backend
python -m venv .venv && .venv\Scripts\activate          # macOS/Linux: source .venv/bin/activate
pip install -r requirements.txt
# optional: keep model weights off the system drive
echo MODEL_CACHE_DIR=D:/MultimediaToolKit/models > .env
python -m app.cli download-models                      # Whisper tiny+base, Kokoro (~330 MB)
uvicorn app.main:app --reload                           # http://localhost:8000/docs

System tools: FFmpeg (required) and Tesseract (for OCR modes; Windows: winget install UB-Mannheim.TesseractOCR). Migrations run automatically at start-up (alembic upgrade head also works). In development, requests without a key act as a local developer user.

Frontend

cd frontend
npm install
set NEXT_PUBLIC_API_URL=http://localhost:8000            # macOS/Linux: export ...
npm run dev                                              # http://localhost:3000

以上为 README 的开头部分。其余内容请阅读原文。 查看完整 README ↗