ESP32-S3 Bench Guide
ESP32-S3 N16R8 bench guide: live pinout, lessons, simulators and the bench log.
翻译准备中
尚未检测到更新。GitHub 发布版本后会显示在这里。
个人 GitHub 账号下公开仓库的所有者可以登录并验证所有权,自行管理介绍,无需管理员批准。
登录并生成验证文件,在 30 分钟内将其提交到此仓库的默认分支。其他分支或分叉仓库中的文件无效。然后在下方检查文件,即可无需管理员批准直接管理项目。请保留文件以维持管理权限。
如需登录,将前往登录页面。登录后请回到此处,再次点击所选的验证按钮。
在项目文件夹中运行 /nomorevibe verify,即可验证所有权并自行管理介绍。
一分钟完成验证An AI video navigation system that allows you to search and ask questions simply by uploading videos.
翻译准备中
取自仓库 README
正在加载完整 README。
查看完整 README ↗Ask your videos. Find the explanation.
VideoQ helps you study with your own lecture videos. Ask a question in your own words, read an answer grounded in your course material, and select a source timestamp to rewatch the relevant explanation.
Upload recordings or import YouTube lectures, organize them into courses, and share them with learners. Course owners can review question history and feedback to see where learners need help.
Try VideoQ · Developer docs · Quick start
▶ Play the English demo (30 seconds) · Watch on the website with captions
The demo follows a learner through a five-subject course: open a lecture, ask what a derivative means, read the AI answer, and click its timestamp to revisit the explanation.
English adaptation of a walkthrough using real course data and an AI answer. Waiting times have been shortened.
New to the team? Start with the developer documentation: local setup, first walkthrough, and codebase tour. Architecture and design references are also available there.
For the research behind the AI pipeline, see AI methods and references (Japanese translation) and the reusable BibTeX bibliography. It maps RAG, ReAct, Whisper, Otsu-inspired subtitle splitting, VideoSeek-inspired navigation, and the configured models to their original papers or official sources.
| Layer | Stack |
|---|---|
| Frontend | React 19, TypeScript, Vite, TanStack Query → Cloudflare Workers Static Assets |
| Web API | Hono + tRPC, Zod, Drizzle ORM → Cloudflare Workers |
| Authentication | Better Auth cookie sessions, API keys, OAuth provider; optional Google sign-in |
| Async jobs | Python worker → Amazon SQS / AWS Lambda |
| Database | Neon PostgreSQL + pgvector through Hyperdrive (local: PostgreSQL 17 + pgvector) |
| Object storage | Cloudflare R2 (local: Garage) |
| Edge state | Durable Objects (rate limits and task recovery scheduling) |
| External services | OpenAI / optional local AI, SearchAPI for YouTube transcripts, Mailgun email, Stripe billing |
Locally, docker compose runs Postgres, Garage, ElasticMQ, the Hono API (wrangler dev), the Python worker, a static frontend build, and a Caddy gateway on port 80.
For on-prem Kubernetes, the Helm chart deploys a production Node.js API, static web and Python workers against operator-provided PostgreSQL, S3 storage, a persistent SQS-compatible queue and configured inference endpoints. See the on-prem guide for the runtime differences.
Browser → Caddy → React (nginx)
→ Hono API → PostgreSQL / ElasticMQ (SQS)
Browser → Garage (signed uploads) ↓
Python worker → PostgreSQL / Garage
The SPA shares the typed /api/trpc contract in packages/trpc. Hono also serves Better Auth, MCP, chat streaming, media, CSV exports, and the Stripe webhook. Browser authentication uses cookies; MCP uses API keys or OAuth access tokens.
Async processing runs transcription → scene indexing. Jobs are recorded in a database outbox and delivered to SQS, and the worker tracks job IDs to handle retries. The TASK_SCHEDULER Durable Object schedules recovery for pending deliveries and abandoned uploads. A daily 17 3 * * * UTC cron performs retention cleanup and recovery; there is no five-minute polling cron.
Package READMEs: apps/ · apps/api/ · apps/web/ · apps/worker/
sk-...The setup below uses OpenAI for transcription, embeddings, and chat. Optional local AI configuration is described below.
# Clone the project and enter the directory
git clone https://github.com/videoq-jp/videoq.git
cd videoq
# Copy the environment file
cp .env.example .env
Open .env and set at least:
OPENROUTER_API_KEY=your-openrouter-api-key
OPENAI_BASE_URL=https://openrouter.ai/api/v1
LLM_MODEL=openai/gpt-6-luna
EMBEDDING_MODEL=openai/text-embedding-3-small
WHISPER_BACKEND=openrouter
WHISPER_BACKEND=openai
EMBEDDING_MODEL=text-embedding-3-small
# Docker API startup currently forwards AUTH_JWT_SECRET as the auth fallback.
# AUTH_JWT_SECRET: openssl rand -base64 48
# USER_SECRET_ENCRYPTION_KEY: openssl rand -base64 32 | tr '+/' '-_' | tr -d '='
AUTH_JWT_SECRET=
USER_SECRET_ENCRYPTION_KEY=
The hosted configuration uses OpenRouter for answers, image inspection, embeddings,
and transcription. Prefer workload identity federation: configure separate
OPENROUTER_FEDERATION_POLICY_ID values for the API and worker, using AWS STS
identities and OpenRouter Business/Enterprise policies. A fixed
OPENROUTER_API_KEY is retained for explicit development compatibility.
The existing GPT-6 Luna and 1536-dimensional text-embedding-3-small models are
preserved; only transcription changes to openai/whisper-large-v3-turbo.
OpenRouter uses qualified openai/... model IDs. The adapters also qualify the
existing unprefixed default model names when the selected endpoint is OpenRouter.
This alias change alone does not require rebuilding existing embedding vectors.
For Cloudflare, configure the API policy ID and AWS_REGION in production
Wrangler vars and grant STS token permission to its existing AWS principal. For
Lambda, set Terraform openrouter_federation_policy_id; it uses its execution
role. See the WIF setup and release runbook.
For Helm, configure ai.federationPolicyId / ai.federationAudience and supply
an AWS identity to the runtime. Local inference remains available through the
local overlays. Without a policy, OPENROUTER_API_KEY remains a development
option. Remove stale endpoint overrides when selecting the shared provider.
A configured federation policy takes precedence over fixed keys on OpenRouter
and never falls back to them after an exchange failure.
Before production use, verify Whisper returns real segment timestamps using
verbose_json, and run the opt-in structured-answer live test. No fabricated
subtitle timing is accepted. See the release runbook,
OpenRouter transcription,
and embeddings.
Answer generation and image inspection share LLM_MODEL, which defaults to GPT-6 Luna (gpt-6-luna). With visual Q&A enabled, the model must support image input as well as tool calling and strict JSON output. See model settings and model references.
Embedding dimensions are fixed at 1536 in the API, worker, and DB. The templates use OpenAI / text-embedding-3-small; no dimension setting is needed. Configure the same provider and model for both runtimes. See embedding configuration and diagnostics.
Docker Compose supplies development-only fallback secrets when these fields are blank. The API container generates apps/api/.dev.vars using docker-dev.sh, which currently forwards AUTH_JWT_SECRET, not BETTER_AUTH_SECRET. Host-run API development and production use BETTER_AUTH_SECRET directly. Configure independent secrets for shared deployments.
# Start all services. The first run may take a few minutes.
# Drizzle migrations run automatically via the `migrate` service.
docker compose up --build -d
Check service state and logs if startup or video processing fails:
docker compose ps -a
docker compose logs --tail=100 migrate api worker
migrate and garage-init are one-shot services; exiting with code 0 is expected. The API provides /health for liveness and /ready for database readiness, both accessible through http://localhost.
Optional Vite HMR for frontend work:
docker compose --profile dev up -d web-dev
# → http://localhost:3000
npm ci
npm run user:admin --workspace @videoq/api -- your-username-or-email
If a migrated local account reports Password not found, restore a local-only
temporary password (the command prints it once):
npm run user:password:local --workspace @videoq/api -- your-username-or-email
Open http://localhost in your browser.
Useful links:
http://127.0.0.1:9000 (bucket status: docker compose exec garage /garage bucket info videoq-media)127.0.0.1:55432http://127.0.0.1:9324 (SQS endpoint), statistics UIFirst steps:
New accounts receive a monthly free tier automatically (override per user later in Admin):
| Setting | Default |
|---|---|
| Max video upload size (MB) | 200 (MAX_VIDEO_UPLOAD_SIZE_MB) |
| Storage limit (GB) | 1 (DEFAULT_STORAGE_LIMIT_GB) |
| Processing limit (minutes / month) | 45 (DEFAULT_PROCESSING_LIMIT_MINUTES) |
| AI answers limit (per month) | 30 (DEFAULT_AI_ANSWERS_LIMIT) |
The DEFAULT_* quota variables accept null or unlimited for no cap; 0 is a hard zero quota. MAX_VIDEO_UPLOAD_SIZE_MB is a numeric upload limit. In Admin, leave nullable quota fields blank to set unlimited. AI answers and processing usage reset each UTC month.
For Docker development, use Admin to change user quotas. API runtime variables come from wrangler.jsonc and the subset written by docker-dev.sh; adding an arbitrary API variable to the root .env does not automatically expose it to Hono. In particular, signup quota overrides are not currently forwarded by that script.
Local default: Docker Compose starts a single-node Garage instance and configures the API + worker to use it. Browser uploads go to http://127.0.0.1:9000; containers use http://garage:3900, with region garage. garage-init applies bucket CORS before the API and worker start. Data and metadata persist in the garage_data volume. See the local storage guide for credentials, inspection, verification, and migration from an existing MinIO stack.
Production: Use Cloudflare R2 (or another S3-compatible store). Set the API secrets / vars described in infra/DEPLOY.md, including R2_* credentials and USE_S3_STORAGE=true for the worker / frontend as needed. Browser uploads also require the versioned R2 CORS policy in apps/api/r2-cors.production.json.
Example API runtime variables / secrets for R2:
USE_S3_STORAGE=true
R2_ACCESS_KEY_ID=your-r2-access-key
R2_SECRET_ACCESS_KEY=your-r2-secret-key
R2_BUCKET_NAME=your-bucket
R2_S3_ENDPOINT=https://<account-id>.r2.cloudflarestorage.com
R2_S3_REGION=auto
The deployed API uses the VIDEO_BUCKET binding for object metadata and deletion. The R2 S3 credentials are used to sign browser URLs and by the Python worker. In Lambda, store R2 credentials under R2_ACCESS_KEY_ID / R2_SECRET_ACCESS_KEY; AWS credential variables belong to the execution role.
The checked-in Compose stack explicitly configures Garage endpoints and credentials. Switching it to external storage requires updating those service settings as well as browser CORS and the publicly reachable storage URL.
After editing runtime values in the root .env, recreate affected containers to load them:
docker compose up -d --force-recreate api worker
Frontend VITE_* values are build-time settings; changes require rebuilding web.
This step is optional. Skip it if the default OpenAI setup works for you.
Transcription, chat, and embeddings can use local services. Configure each component together with the API and worker that consume it.
Run whisper.cpp natively on macOS to use Apple Silicon's Metal backend.
The on-prem Kubernetes recipes use whisper.cpp on CPU or vLLM on NVIDIA GPUs,
behind a private transcription gateway.
Both connect through WHISPER_BACKEND=local. Build the pinned submodule with
CMake and a C++ toolchain; install FFmpeg on the Mac for MP3 conversion.
Quick setup:
# 1. Fetch whisper.cpp from the VideoQ root directory
git submodule update --init --recursive
cd whisper.cpp
# 2. Build
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --config Release
# 3. Download a model
bash ./models/download-ggml-model.sh large-v3-turbo
# 4. Start the server
./build/bin/whisper-server -m models/ggml-large-v3-turbo.bin \
--host 127.0.0.1 --port 8080 --inference-path /audio/transcriptions \
--convert -l auto
--convert is required for the worker's MP3 uploads. Keep timestamps enabled.
Use -l ja instead of -l auto to fix recognition to Japanese. The server loads
the selected local model at startup; it does not download a model per request.
Configure VideoQ:
In another terminal at the VideoQ root, edit .env:
WHISPER_BACKEND=local
WHISPER_LOCAL_URL=http://host.docker.internal:8080
Recreate the worker to load the changed environment:
docker compose up -d --force-recreate worker
For a worker running directly on the Mac, use http://127.0.0.1:8080 instead.
The base URL has no /v1 suffix for this whisper.cpp command. Keep this
unauthenticated development server bound to loopback. See the
local inference guide
for endpoint differences and the real-audio acceptance check.
Use llama.cpp for the default CPU recipe or vLLM for NVIDIA GPU inference. Chat and embeddings use separate private OpenAI-compatible APIs; transcription uses the private whisper.cpp/vLLM gateway. See the local inference guide and the inference chart for matching application overlays, model provisioning and acceptance commands.
Set EMBEDDING_BASE_URL and the matching served model alias in both API and worker.
OPENAI_BASE_URL selects chat independently. Private servers without
API authentication use local-inference as a placeholder key. Text-only models
require VIDEO_VISUAL_ENABLED=false; hosted credentials are unnecessary when
chat, embeddings and Whisper all use validated private services.
Changing embedding models requires maintenance, backups and complete vector regeneration before resuming search, even when dimensions remain 1536. See diagnostics and migration constraints.
See the workflow map for CI, deployment, automatic dependency updates, and operations checks.
Run npm commands from the repository root. Dependencies for apps/api, apps/docs, apps/web, and packages/trpc share the root lockfile.
npm ci
npm run typecheck
npm run lint
npm test
npm run build
npm test runs API unit / Workers runtime tests and frontend tests. API database integration tests additionally require QUOTA_TEST_DATABASE_URL pointing to a test PostgreSQL instance with pgvector and permission to create test databases.
Read the published docs in English or Japanese.
The Docusaurus site reads English Markdown from docs/ and Japanese
translations from apps/docs/i18n/ja/docusaurus-plugin-content-docs/current/.
English is the default at /; the header language menu switches to Japanese at /ja/.
Both languages include Mermaid diagrams and local full-text search.
Documentation changes merged into main are automatically published by
CD after CI succeeds.
npm run dev:docs # http://localhost:3001
npm run dev:docs -- --locale ja # Japanese only: http://localhost:3001/ja/
npm run build:docs # Both languages: apps/docs/build
npm run preview:docs # Preview the build, including search, on port 3001
npm run deploy:docs # Build and publish the current working tree to docs.videoq.jp
The development server runs one language at a time. Build and preview to check
the language switcher and search; search indexes are generated during the build. See
apps/docs/README.md for editing and hosting settings.
For frontend-only work against the Compose API:
VITE_API_URL=/api VITE_USE_S3_STORAGE=true npm run dev:web
# http://localhost:3000; Vite proxies /api to 127.0.0.1:8787
To also run Hono outside Docker, stop the Compose API to release port 8787, then prepare Wrangler's variables:
docker compose stop api
cp apps/api/.dev.vars.example apps/api/.dev.vars
Set BETTER_AUTH_SECRET, USER_SECRET_ENCRYPTION_KEY, and OPENROUTER_API_KEY in apps/api/.dev.vars. For the hosted setup above, also set:
BETTER_AUTH_URL=http://localhost:3000
FRONTEND_URL=http://localhost:3000
OAUTH_ISSUER_URL=http://localhost:3000
OPENAI_BASE_URL=https://api.openai.com/v1
EMBEDDING_MODEL=text-embedding-3-small
Keep the Garage / ElasticMQ settings from .dev.vars.example, use the same user-secret encryption key as the Python worker, then run npm run dev:api in another terminal. Wrangler's local Hyperdrive connection targets localhost:55432; override it if you changed the database credentials. The Compose dependencies and Python worker continue running.
Restarting the Compose API regenerates .dev.vars, so keep host-only configuration separately if you switch between these workflows. MCP OAuth discovery additionally needs /.well-known/* forwarding, as shown in Caddyfile.dev; Vite currently proxies /api, /health, and /ready only.
Edit apps/api/src/db/schema/, then generate and verify migrations:
npm run db:generate -- --name describe_the_schema_change
npm run db:check
npm run db:verify
npm run db:migrate
Do not hand-edit generated DDL SQL, snapshots, or the journal. Data-only backfills use npm run db:generate:custom -- --name describe_the_data_change and must start with -- drizzle-kit:custom; custom migrations must not contain DDL. db:verify checks migration provenance, and the API database integration tests apply the full migration history to an empty database.
db:migrate uses DATABASE_URL when exported, otherwise the local postgres:postgres connection on port 55432. Root .env values are not automatically loaded by Drizzle Kit. See the API README for details.
cd apps/worker
python3.12 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
python -m pytest tests/ -q
PostgreSQL integration tests additionally use DATABASE_URL. Running transcription outside Docker requires FFmpeg and the database, storage, queue, and AI environment settings described in the worker README.
Dependabot checks for version updates every Monday at
09:00 Asia/Tokyo. It covers the root npm workspaces, the Python worker's
pyproject.toml / uv.lock, GitHub Actions, Dockerfiles, Docker Compose,
Terraform (including infra/bootstrap), and the optional whisper.cpp submodule.
Related npm packages are updated together; remaining minor and patch updates are
grouped by ecosystem, while major updates are reviewed separately. GitHub Actions
updates share one group. Version-update PRs are limited to five per ecosystem
(one for the submodule).
The configuration takes effect on GitHub's default branch (main). Dependabot
alerts and security updates are separate repository settings under
Settings → Advanced Security; keep both enabled. npm and Python security fixes
are grouped separately from scheduled version updates. Verified minor and patch
updates for npm, Python, Docker, Compose, Terraform providers, and SHA-pinned
GitHub Actions are automatically merged after required checks pass. Major updates,
submodules, deployment tools, and source changes require review. Terraform provider
minor/patch updates also apply automatically after merge; ordinary infrastructure
changes retain production approval. See dependency automation.
CI and production worker builds install hash-verified binary dependencies from uv.lock.
The production topology is Cloudflare Workers (separate frontend, API, and docs) + Hyperdrive, Neon PostgreSQL, Cloudflare R2, and an ARM64 Python Lambda consuming SQS. Follow infra/DEPLOY.md for provisioning, secrets, migrations, CORS, and deployment checks.
videoq-web Worker + Static Assets, configured in apps/web/wrangler.jsonc; GitHub Actions CD deploys after successful main CI.infra/ manages AWS resources.BETTER_AUTH_SECRET, public URLs, and Mailgun for verification / invitation email; Google sign-in is optional.Caddy accepts a public hostname through SITE_ADDRESS, but the checked-in stack runs wrangler dev with development settings and signs uploads for http://127.0.0.1:9000. Setting a domain in .env alone does not configure a public deployment: authentication origins, CORS, API variable forwarding, secrets, and a browser-accessible HTTPS storage endpoint also need configuration. Use the production deployment guide above for the maintained cloud setup.
VideoQ exposes a remote MCP server at /api/mcp (Streamable HTTP via @hono/mcp). Connect a compatible client using the endpoint URL and an API key, or use the OAuth authorization flow. Older clients can use the mcp-remote bridge below.
🛡️ Permission policy: Read tools accept
read_onlyAPI keys or OAuth tokens withvideoq.read. Video upload and course management tools require anallAPI key or the OAuthvideoq.writescope. Missing API-key permission metadata defaults to read-only. Sending RAG chat questions is intentionally excluded from MCP.
VideoQ exposes 9 read-only tools and 5 write tools. The MCP metadata and tests enforce that split, including each tool's input/output schema and safety annotations.
| Tool | Purpose |
|---|---|
list_videos / get_video | List videos and view details (with opt-in, bounded transcript chunks) |
request_video_upload / confirm_video_upload | Upload a local video through a signed object-storage URL and start processing |
create_youtube_video | Register and import a YouTube lecture URL |
list_courses / get_course | List courses and their member videos |
create_course | Create a course |
add_video_to_course | Add an uploaded video to a course |
list_tags | List tags |
get_chat_history | Chat history for a course (with feedback) |
get_chat_analytics | Question counts, period, daily time series, feedback aggregates |
List tools support limit / offset pagination. General lists default to 20 and allow at most 100 items; chat history defaults to 10 and allows at most 25. get_course paginates member videos separately with video_limit / video_offset. get_video omits the transcript by default; request bounded chunks with include_transcript, transcript_offset, and transcript_limit.
The three create/reserve tools (request_video_upload, create_youtube_video, and create_course) require an idempotency_key. Generate one stable value per logical operation and reuse it only when retrying the same arguments. Upload confirmation and course membership addition are also safe to retry.
Log in to VideoQ and issue a vq_... key from Settings → Integration API Keys, then copy it. Select all to upload videos or manage courses; read_only is enough for analytics.
The endpoint URL is your VideoQ host followed by /api/mcp — for example, http://localhost/api/mcp for a local Docker setup or https://your-domain.example.com/api/mcp in production. Authenticate with Authorization: Bearer vq_... (or the equivalent X-API-Key header).
For Claude Code:
claude mcp add --transport http videoq https://your-domain.example.com/api/mcp \
--header "Authorization: Bearer vq_xxxxxxxxxxxxxxxx"
For Claude Desktop / claude.ai (built-in connector, OAuth 2.1), paste just the MCP URL into Settings → Connectors → Add custom connector and approve the consent screen. No API key needed — VideoQ implements OAuth 2.1 + Dynamic Client Registration (RFC 7591) per the MCP Authorization spec.
https://your-domain.example.com/api/mcp
The client discovers the authorization server through /.well-known/oauth-protected-resource/api/mcp and /.well-known/oauth-authorization-server/api/auth, registers at /api/auth/oauth2/register, and runs the authorization-code flow with PKCE. The authorization server issuer is {public-origin}/api/auth, and the protected resource is {public-origin}/api/mcp. VideoQ enforces videoq.read / videoq.write scopes. Revoke grants from Settings → Connected Apps.
For your own production instance, complete the deployment setup. Set the API runtime URL variables to the public HTTPS origin:
BETTER_AUTH_URL=https://videoq.example.com
FRONTEND_URL=https://videoq.example.com
OAUTH_ISSUER_URL=https://videoq.example.com
After deployment, confirm that the MCP endpoint and OAuth metadata are reachable before registering the connector (an unauthenticated MCP request can return 401):
https://videoq.example.com/api/mcp
https://videoq.example.com/.well-known/oauth-authorization-server/api/auth
https://videoq.example.com/.well-known/oauth-protected-resource/api/mcp
If you need to fall back to the mcp-remote bridge for an older client, configure it with your API key instead:
{
"mcpServers": {
"videoq": {
"command": "npx",
"args": [
"mcp-remote",
"https://your-domain.example.com/api/mcp",
"--header",
"Authorization: Bearer vq_xxxxxxxxxxxxxxxx"
]
}
}
}
Config file locations:
~/Library/Application Support/Claude/claude_desktop_config.json%APPDATA%\Claude\claude_desktop_config.jsonCursor and other Streamable HTTP-capable clients use the same URL + header pattern.
Restart the client and confirm that the MCP server appears as videoq. Try prompts like "Create a course named Physics", "Upload lecture.mp4 and add it to the Physics course", or "Analyze the chat history for course 1".
For a local file, Claude Code generates an idempotency key, calls request_video_upload, uploads the file with an HTTP PUT using the returned URL and required headers, and then calls confirm_video_upload. The video bytes do not pass through the Worker or MCP JSON payload. Retrying the prepare call after confirmation returns the current video state without minting an overwrite-capable PUT URL.
401 Unauthorized → The API key (or OAuth token) is missing, malformed, or revoked. Reissue from Settings and update the header, or re-approve the OAuth connector.404 Not Found → The URL is wrong. Confirm the host and the /api/mcp path.https://<host>/.well-known/oauth-authorization-server/api/auth and https://<host>/.well-known/oauth-protected-resource/api/mcp return JSON. Reverse proxies must forward /.well-known/* to Hono, and metadata must advertise the public origin.apps/api/ Hono + tRPC on Cloudflare Workers
apps/web/ React SPA
apps/worker/ Python async pipeline (SQS / Lambda)
packages/trpc/ Shared Hono ↔ React tRPC router and contract
package.json npm workspace root
infra/ Terraform (SQS, Lambda, ECR, IAM) + deploy notes
docs/ Architecture and design docs
whisper.cpp/ Optional local transcription server (Git submodule)
.github/workflows/ CI, CD, Terraform, Cloudflare resource management
docker-compose.yml Local full stack
Caddyfile.dev Local gateway and protocol routing
Found a bug or want to add a feature? Contributions are welcome.
For frontend UI changes, follow the Storybook change and review workflow to update the relevant stories and share verification results in your PR.
See the LICENSE file for details.