SolomindLM

AI-powered research platform for multi-source content ingestion, RAG-based chat, and automated content generation.

---

Table of Contents

• Overview
• Features
• Architecture
• Tech Stack
• Prerequisites
• Quick Start
• Environment Variables
• Development
• Testing
• Deployment
• Contributing
• License

---

Overview

SolomindLM is an open-source AI research platform that helps you ingest content from multiple sources, chat with your documents using RAG (Retrieval-Augmented Generation), and automatically generate structured content like reports, flashcards, quizzes, and more.

Why SolomindLM?

• All-in-one research workspace — Collect sources, chat with them, and generate content in one place
• Multi-source ingestion — PDFs, images, audio, web pages, YouTube videos, social media posts
• RAG-powered chat — Ask questions and get answers with citations from your sources
• Automated content generation — Turn your research into reports, flashcards, quizzes, mind maps, infographics, and more

---

Features

Content Ingestion

• Documents — Upload PDFs, Word docs, images, and audio files
• Paper import — BibTeX, Zotero, Mendeley, and DOI-based bibliography import
• Academic discovery — Search Semantic Scholar, PubMed, and arXiv from the sources UI
• Web scraping — Extract content from any web page
• Social media — Import transcripts from YouTube, TikTok, Instagram, and X (Twitter)
• Web search — Search and add sources via Tavily
• Google Drive — Import files directly from Google Drive
• Source guide — AI-generated overview of ingested source material

RAG Chat

• Streaming conversations — Real-time AI responses with smooth streaming
• Citations — Every answer includes references to source materials
• Source @mentions — Reference specific documents in chat prompts
• Deep research — Multi-step research plans with evidence gathering and synthesis
• Literature review — Systematic screening, ranking, and synthesis of academic papers
• External search — Augment responses with web, academic, and news search
• Voice input — Speech-to-text for chat messages

Content Generation (Studio)

• Reports — Comprehensive research reports with structured sections
• Flashcards — Study cards with questions and answers
• Quizzes — Multiple-choice and open-ended questions
• Mind maps — Visual knowledge maps
• Infographics — Visual summaries generated via Together AI image models
• Audio overviews — Podcast-style summaries via text-to-speech
• Spreadsheets — Structured data tables
• Written questions — Short answer and essay prompts
• Literature tables & reports — Structured outputs from literature review workflows
• Prompt library — Curated, community-rated studio prompts (see PROMPT_LIBRARY.md)

Organization

• Notebooks — Organize research into projects
• Folders — Group notebooks hierarchically
• Notes — Saved chat excerpts and manual notes per notebook
• Sharing — Collaborate via share links, members, or fork-only links
• Onboarding — Guided first-run experience for new users
• Output language — Per-user preference for generated content language
• History — Persistent chat history and generated content

---

Architecture

┌─────────────────────────────────────────────────────────────┐
│                        Clients                               │
│  ┌──────────────┐  ┌──────────────┐  ┌─────────────────┐   │
│  │   Web App    │  │  Mobile App  │  │   E2E Tests     │   │
│  │  (React 19)  │  │  (Expo 55)   │  │  (Playwright)   │   │
│  └──────┬───────┘  └──────┬───────┘  └─────────────────┘   │
└─────────┼─────────────────┼──────────────────────────────────┘
          │                 │
          └────────┬────────┘
                   │
┌──────────────────▼──────────────────────────────────────────┐
│                    Convex Backend                            │
│                                                              │
│  ┌──────────────┐  ┌──────────────┐  ┌─────────────────┐   │
│  │   Auth       │  │    API       │  │   Scheduler     │   │
│  │(Google OAuth)│  │(Queries/Actions)│  │ (Background    │   │
│  │(Password+OTP)│  │              │  │   Jobs)         │   │
│  └──────────────┘  └──────┬───────┘  └─────────────────┘   │
│                           │                                  │
│  ┌──────────────┐  ┌──────▼───────┐  ┌─────────────────┐   │
│  │   Storage    │  │   Agents     │  │   Vector DB     │   │
│  │ (Documents,  │  │ (Phased jobs,│  │ (Document       │   │
│  │  Notebooks)  │  │  Workflows)  │  │  Chunks)        │   │
│  └──────────────┘  └──────────────┘  └─────────────────┘   │
└─────────────────────────────────────────────────────────────┘
          │
          └────────┬────────┬────────┬────────┬────────┐
                   │        │        │        │        │
┌──────────────────▼──┐ ┌───▼───┐ ┌──▼───┐ ┌─▼────┐
│    Together AI      │ │Tavily │ │Mistral│ │Stripe│
│ (LLMs, Embeddings,  │ │Search │ │ OCR   │ │Billing│
│  TTS, Images)       │ │       │ │       │ │       │
└─────────────────────┘ └───────┘ └───────┘ └──────┘
┌──────────────────┐  ┌───────────┐  ┌──────────────────────┐
│   Supadata       │  │ Voyage AI │  │      Resend          │
│ (Extraction)     │  │(Reranking)│  │    (Email OTP)       │
└──────────────────┘  └───────────┘  └──────────────────────┘

Data Flow

1. Ingestion → Document upload → OCR/Extraction → Smart splitting → Embeddings → Vector store
2. Chat → User query → Vector search → Reranking → LLM with context → Streaming response
3. Studio generation → User request → Job row → Phased Convex actions (single pass, or parallel map-chunk actions then a finalize phase) → Result table → Reactive query delivery
4. Deep research & literature review → User request → @convex-dev/workflow durable workflow → User approval of the plan → Retrieval and writing → Result (deep research streams tokens)

LangChain supplies the Together AI chat client, message types and text splitting. Earlier versions ran the studio agents as LangGraph StateGraphs; they were replaced by phased actions to stay inside Convex's action time limit, and @langchain/langgraph is no longer a dependency.

---

Tech Stack

Layer	Technology
Frontend	React 19, Vite 7, TypeScript 5.9, TailwindCSS 4, Streamdown + KaTeX
Mobile	Expo 55, React Native 0.83 (WebView shell + native auth/push)
Backend	Convex 1.36, TypeScript
AI/ML	Together AI, LangChain, @convex-dev/workflow
Auth	@convex-dev/auth (Google OAuth + Password/OTP)
Payments	Stripe
Database	Convex (Document + Vector search)
Search	Tavily, Voyage AI reranking
OCR	Mistral
Extraction	Supadata
Audio	Together AI TTS
Testing	Vitest, Playwright, convex-test

---

Prerequisites

Before you begin, ensure you have:

1. Bun v1.2.2 or higher

   curl -fsSL https://bun.sh/install | bash
   

2. Node.js v20.0.0 or higher (for compatibility)

3. Git

4. A Convex account (free tier available)

5. API keys for the services you want to use (see Environment Variables)

---

Quick Start

1. Clone the Repository

git clone https://github.com/samintisar/SolomindLM.git
cd SolomindLM

2. Install Dependencies

bun install

3. Set Up Convex

# Initialize Convex (creates a new project)
bun x convex dev

# This will:
# - Create a Convex project
# - Start the Convex dev server
# - Generate the Convex client code

4. Configure Environment Variables

Backend (Convex):

# Copy the example environment file (dev uses .env.local)
cp .env.example .env.local

# Edit .env.local and add your API keys
# See Environment Variables section below for details
# Pull from Convex: bun run convex:env:pull:dev

Frontend (Web):

# Copy the web environment file
cp apps/web/.env.local.example apps/web/.env.local

# Edit apps/web/.env.local with your Convex URLs

Frontend (Mobile - optional):

# Copy the mobile environment file
cp apps/mobile/.env.local.example apps/mobile/.env.local

5. Push Environment Variables to Convex

bun run convex:env:push

6. Start the Development Servers

Terminal 1 — Convex backend:

bun run dev:convex

Only one checkout can watch the dev deployment at a time. In other git worktrees, push changes once with bun run dev:convex --once.

Terminal 2 — Web frontend:

bun run dev:web

Terminal 3 — Mobile (optional):

bun run dev:mobile

Open the URL the dev server prints: http://localhost:5173 in the main checkout. Each git worktree gets its own port (5174-5199) so several can run at once.

---

Environment Variables

Required Variables

Create .env.local in the project root for dev (prod uses .env). See .env.example for the template. Use bun run convex:env:pull:dev to sync from your Convex dev deployment.

Convex (Required)

Variable	Description	How to Get
CONVEX_DEPLOYMENT	Your Convex deployment name	Created automatically by convex dev
CONVEX_URL	Your Convex deployment URL	Found in Convex dashboard
CONVEX_SITE_URL	Your Convex HTTP site URL	Found in Convex dashboard
SITE_URL	Single canonical site origin (Convex Auth builds email links from it — never comma-joined)	Your domain
CORS_EXTRA_ORIGINS	Optional. Extra origins allowed for CORS + auth redirects, comma-separated	e.g. apex domain alongside www, staging

Authentication (Required on Convex)

Variable	Description	How to Get
AUTH_GOOGLE_ID	Google OAuth client ID (Convex)	Google Cloud Console
AUTH_GOOGLE_SECRET	Google OAuth client secret (Convex)	Google Cloud Console
JWKS	Public JWKS JSON for session tokens	Convex Auth setup
JWT_PRIVATE_KEY	Private key matching JWKS	Convex Auth setup

AI Services (Required for core functionality)

Variable	Description	How to Get
TOGETHER_AI_API_KEY	Together AI API key	Together AI
TAVILY_API_KEY	Tavily search API key	Tavily
MISTRAL_API_KEY	Mistral OCR API key	Mistral
SUPADATA_API_KEY	Supadata extraction API key	Supadata
VOYAGE_API_KEY	Voyage AI reranking API key (rerank-3)	Voyage AI
SEMANTIC_SCHOLAR_API_KEY	Semantic Scholar (optional)	Semantic Scholar
PUBMED_EMAIL	PubMed API contact (optional)	Your team email
OPEN_ACCESS_CONTACT_EMAIL	Unpaywall / OpenAlex lookup contact (optional; defaults to [email protected])	Your team email

Optional Services

Variable	Description	Required For
STRIPE_SECRET_KEY	Stripe secret key	Billing/subscriptions
STRIPE_WEBHOOK_SECRET	Stripe webhook secret	Billing webhooks
STRIPE_PRO_MONTHLY_PRICE_ID	Stripe price ID	Pro plan (monthly)
STRIPE_PRO_YEARLY_PRICE_ID	Stripe price ID	Pro plan (yearly)
RESEND_API_KEY	Resend email API key	Email OTP/password reset
AUTH_RESEND_FROM	From email address	Email sending
LANGCHAIN_API_KEY	LangSmith API key (local only, not pushed)	Tracing when running CLI/tools locally
LANGCHAIN_PROJECT	LangSmith project name	Tracing organization

Audio TTS (Optional on Convex)

Variable	Description
AUDIO_VOICE_HOST_A	Together Kokoro voice ID for host A (default af_sky)
AUDIO_VOICE_HOST_B	Together Kokoro voice ID for host B (default am_echo)

RAG evaluation (dev only)

Variable	Where	Description
RAG_EVALS_ENABLED	Convex dev	Set true to enable eval actions
RAG_EVAL_SECRET	Convex dev + local .env	Shared secret (≥16 chars); see evals/rag/env.eval.example
RAG_EVAL_CONVEX_URL	Local only	Dev deployment URL for bun run eval:rag (not pushed)

Run bun run eval:rag:bootstrap-env once to wire dev URL + secret.

LLM Model Configuration

You can customize which models are used for different features. Models are hosted on Together AI using their published model identifiers:

FAST_LLM=Qwen/Qwen3.5-9B
SMART_LLM=deepseek-ai/DeepSeek-V4.1-Flash
REPORT_LLM=deepseek-ai/DeepSeek-V4.1-Flash
FLASHCARDS_LLM=deepseek-ai/DeepSeek-V4.1-Flash
QUIZ_LLM=deepseek-ai/DeepSeek-V4.1-Flash
MINDMAP_LLM=deepseek-ai/DeepSeek-V4.1-Flash
SPREADSHEET_LLM=deepseek-ai/DeepSeek-V4.1-Flash
WRITTEN_QUESTIONS_LLM=deepseek-ai/DeepSeek-V4.1-Flash
AUDIO_LLM=deepseek-ai/DeepSeek-V4.1-Flash

Frontend Environment Variables

Create apps/web/.env.local (see apps/web/.env.local.example):

VITE_CONVEX_URL=https://your-deployment.convex.cloud
VITE_CONVEX_SITE_URL=https://your-deployment.convex.site
VITE_STRIPE_PUBLISHABLE_KEY=pk_test_...  # Optional, for billing
# Google Drive picker (optional):
# VITE_GOOGLE_CLIENT_ID=...
# VITE_GOOGLE_BROWSER_API_KEY=...
# VITE_GOOGLE_APP_ID=...

Sync from root after pull: bun run convex:env:pull:dev updates CONVEX_URL → VITE_CONVEX_URL when apps/web/.env.local exists.

Mobile Environment Variables

Create apps/mobile/.env.local from apps/mobile/.env.local.example:

EXPO_PUBLIC_CONVEX_URL=https://your-deployment.convex.cloud
EXPO_PUBLIC_WEB_URL=https://your-web-app.com

---

Development

Project Structure

SolomindLM/
├── apps/
│   ├── web/                    # React frontend
│   │   ├── src/
│   │   │   ├── features/       # Domain modules
│   │   │   │   ├── auth/       # Authentication + output language
│   │   │   │   ├── audio/      # Audio playback utilities
│   │   │   │   ├── billing/    # Subscription management
│   │   │   │   ├── chat/       # RAG chat, deep research, literature review
│   │   │   │   ├── landing/    # Marketing pages
│   │   │   │   ├── legal/      # Terms, privacy
│   │   │   │   ├── notebooks/  # Notebook management
│   │   │   │   ├── onboarding/ # First-run onboarding
│   │   │   │   ├── sources/    # Content ingestion + paper import
│   │   │   │   └── studio/     # Content generation views
│   │   │   └── shared/         # UI components, hooks, utils
│   │   └── public/             # Static assets
│   └── mobile/                 # Expo WebView shell (native auth, push)
│       └── app/                # File-based routing
├── convex/                     # Convex backend
│   ├── _agents/                # Agent prompts, routing, heuristics
│   │   ├── _shared/            # Shared agent utilities
│   │   ├── chat/               # Chat agent
│   │   ├── report/             # Report generation
│   │   ├── flashcard/          # Flashcard generation
│   │   ├── quiz/               # Quiz generation
│   │   ├── mindmap/            # Mind map generation
│   │   ├── spreadsheet/        # Spreadsheet generation
│   │   ├── written_questions/  # Written question generation
│   │   ├── audio_overview/     # Audio overview generation
│   │   ├── research/           # Deep research agent
│   │   └── literature_review/  # Literature review agent
│   ├── _lib/                   # Utilities, errors, logging
│   ├── _model/                 # Data models
│   ├── _services/              # External integrations
│   │   ├── ai/                 # AI/LLM services
│   │   ├── search/             # Search services
│   │   ├── extraction/         # Content extraction
│   │   ├── processing/         # Document processing
│   │   ├── grading/            # Content grading
│   │   └── cache/              # Caching logic
│   ├── auth.ts                 # Auth configuration (must be at root)
│   ├── chat/                   # Conversations, messages, voice
│   ├── documents/              # Ingestion, chunks, bibliography
│   ├── literatureReview/       # Literature review workflows
│   ├── notebooks/              # Notebooks, sharing, fork
│   ├── notes/                  # Saved chats + manual notes
│   ├── onboarding/             # User onboarding state
│   ├── push/                   # Mobile push tokens
│   ├── research/               # Deep research plans/runs
│   ├── schema.ts               # Database schema
│   ├── studio/                 # Generation jobs per content type
│   ├── storage/                # Vector store, chat history
│   └── userPreferences/        # Output language, etc.
├── docs/superpowers/           # Internal design specs (not user docs)
├── e2e/                        # Playwright E2E tests
├── evals/                      # RAG + studio eval suite (rag/, ragas/)
├── flowcharts/                 # Diagrams of the original LangGraph agents (historical)
└── PROMPT_LIBRARY.md           # Curated studio prompt seeds

Available Scripts

Root level:

bun run dev              # Start all dev servers
bun run dev:web          # Start web dev server only
bun run dev:mobile       # Start mobile dev server only
bun run build            # Build all apps
bun run build:prod       # Production web build

# Type checking
bun run typecheck:web
bun run typecheck:convex
bun run typecheck:mobile

# Code quality
bun run lint
bun run lint:fix
bun run format
bun run format:check

# Testing
bun run test:convex      # Run Convex unit tests
bun run test:web         # Run web unit tests
bun run test:e2e         # Run Playwright E2E tests

# Convex env sync
bun run convex:env:pull        # Pull Convex env → local files
bun run convex:env:pull:dev    # Dev only → .env.local
bun run convex:env:pull:prod   # Prod only → .env
bun run convex:env:push        # Push .env.local → Convex dev
bun run convex:env:push:prod   # Push .env → Convex prod
bun run convex:env:push:dry    # Dry run env push

# E2E cleanup (Convex test data): delete a test user's `e2e-…` notebooks
bunx convex run e2e/cleanupNotebooks:deleteE2eNotebooksByEmail '{"email":"<test user>"}'

Web app:

cd apps/web
bun run dev              # Start dev server
bun run build            # Build for production
bun run test             # Run unit tests
bun run test:coverage    # Run tests with coverage

Adding a New Feature

1. Create a new feature directory under apps/web/src/features/
2. Add Convex functions under convex/ (or convex/_agents/ for AI features)
3. Write tests for Convex functions in convex/ (.test.ts files)
4. Add E2E tests in e2e/ if UI-facing
5. Update relevant documentation

---

Testing

Unit Tests

Convex tests (uses vitest + convex-test, ~990+ tests):

bun run test:convex
bun run test:convex:watch   # Watch mode
bun run test:integration    # Integration tests only

Web tests (uses vitest):

bun run test:web

E2E Tests

# Create .env.e2e file first
cp .env.e2e.example .env.e2e
# Edit with test credentials

# Run E2E tests
bun run test:e2e

# Run with UI
bun run test:e2e:ui

RAG Evaluation

# One-shot bootstrap (dev URL + secret)
bun run eval:rag:bootstrap-env

# Full eval suite
bun run eval:rag

# Scoped runs (preferred for iteration)
bun run eval:rag -- --case=chat-basic
bun run eval:rag -- --runner=report

# Studio tools (cross-cutting)
bun run eval:studio
bun run eval:literature-review

# Dry runs (no Convex calls)
bun run eval:rag:dry
bun run eval:studio:dry

# Python RAGAS metrics (requires conda env)
bun run eval:ragas

See evals/rag/env.eval.example, evals/rag/fixtures/, and evals/ragas/README.md for setup and runner details.

---

Deployment

Deploying to Vercel (Recommended)

1. Connect your repo to Vercel
2. Set root directory to apps/web
3. Set environment variables in Vercel dashboard:
   • VITE_CONVEX_URL
   • VITE_CONVEX_SITE_URL
   • VITE_STRIPE_PUBLISHABLE_KEY (optional)
4. Add CONVEX_DEPLOY_KEY for production (from Convex dashboard)

The vercel.json in apps/web/ handles:

• Production: convex deploy + web build (deploys backend and frontend together)
• Preview: web build only (avoids overwriting prod Convex — matches CI)
• Monorepo install from repo root (bun install --frozen-lockfile)
• API proxy (/api/* → Convex HTTP actions)
• SPA fallback and security headers
• SEO prerender and sitemap generation (via web build script)

Manual Deployment

# Deploy Convex functions
bun x convex deploy

# Build web app
bun run build:prod

# Deploy to your hosting platform

---

Contributing

We welcome contributions! Please see CONTRIBUTING.md for details.

Quick Contributing Guide

1. Fork the repository
2. Create a branch (git checkout -b feature/amazing-feature)
3. Make your changes
4. Run tests (bun run test:convex && bun run test:web)
5. Commit (git commit -m 'feat: add amazing feature')
6. Push (git push origin feature/amazing-feature)
7. Open a Pull Request

Development Tips

• Follow the existing code style (enforced by Biome via bun run lint / bun run format)
• Write tests for new Convex functions
• Update documentation for new features
• Use conventional commits (feat:, fix:, docs:, refactor:, chore:)

---

Troubleshooting

Common Issues

Build fails with "Cannot find module '@convex/...'"

• Run bun run dev:convex --once to regenerate the Convex client

Vite cache issues

rm -rf apps/web/node_modules/.vite

Port already in use

dev:web / dev:mobile stop this checkout's own leftover server automatically and otherwise move to the next free port. They never kill another checkout's server. If you pinned a port (WEB_PORT, PORT, METRO_PORT) that another process holds, stop that process by its PID or drop the pin.

Convex environment variables not updating

bun run convex:env:push

Type errors in generated files

# Regenerate Convex types (one push; safe while another worktree runs the watcher)
bun run dev:convex --once

Getting Help

• 📖 Convex Documentation
• 📖 React Documentation
• 📖 LangChain Documentation
• 🐛 Open an Issue

---

License

This project is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0) — see LICENSE.

What this means:

• Free for personal use ✅
• Free for self-hosting ✅
• Free for internal business use ✅
• Must open-source modifications if you provide it as a network service

Commercial License Available:

If you want to use SolomindLM in a proprietary SaaS product without open-sourcing your code, a commercial license is available. See COMMERCIAL-LICENSE.md for details.

Open Source vs Commercial Use

Use Case	License Required
Personal/self-hosted	AGPL-3.0 (free)
Internal business use	AGPL-3.0 (free)
Academic/research	AGPL-3.0 (free)
Open-source SaaS	AGPL-3.0 (free)
Proprietary SaaS	Commercial License
White-label/resale	Commercial License

Contact: [email protected] for commercial licensing inquiries (see COMMERCIAL-LICENSE.md).

---

Acknowledgments

• Built with Convex for the backend
• AI powered by Together AI (LLMs, embeddings, TTS, infographics)
• Search powered by Tavily
• OCR powered by Mistral
• Reranking powered by Voyage AI

---