SensAI

Emotion-aware AI tutor that adapts to how you feel.

SensAI detects your facial emotions in real-time through your webcam and adapts its teaching style accordingly. Confused? It simplifies. Bored? It challenges. Frustrated? It encourages. All processing happens in your browser — zero data leaves your device.

Features

• Real-Time Emotion Detection — Custom ResNet-style CNN detects 7 emotions via webcam
• Adaptive Teaching — Pacing, complexity, and tone adjust based on your emotional state
• Learning Analytics — Emotion timeline tracks your journey through each lesson
• Privacy First — All inference runs client-side via WebAssembly. No video data is transmitted
• Multiple Subjects — Python, AI/ML, Mathematics, and more
• No Sign-Up — Open the app and start learning immediately

Tech Stack

Layer	Technology
Frontend	Next.js 15, React 19, TypeScript, TailwindCSS
Emotion Model	Custom ResNet + CBAM CNN (PyTorch) exported to ONNX
Browser Inference	onnxruntime-web (WebAssembly)
Charts	Recharts
Animations	Framer Motion
Icons	Lucide React

Emotion Detection Model

Architecture: SensAIEmotionCNN v2

A residual CNN with CBAM (Convolutional Block Attention Module) attention designed for real-time browser inference.

Input (1x48x48 grayscale)
  |
  +--> Stem: Conv3x3 (1 -> 64) + BN + ReLU
  |
  +--> Stage 1: 2x ResidualBlock 64   (48x48 -> 24x24)  [skip connections]
  +--> Stage 2: 2x ResidualBlock 128  (24x24 -> 12x12)  [skip + CBAM attention]
  +--> Stage 3: 2x ResidualBlock 256  (12x12 -> 6x6)    [skip + CBAM attention]
  |
  +--> Global Average Pooling (6x6 -> 1x1)
  +--> Dropout(0.5) -> FC(256, 128) -> ReLU -> BN -> Dropout(0.25) -> FC(128, 7)
  |
Output (7 emotion probabilities)

v1 vs v2 Comparison

Spec	v1	v2
Architecture	Plain ConvBlocks	Residual Blocks + skip connections
Parameters	1,227,371 (1.2M)	2,833,103 (2.8M)
Attention	SE-only on blocks 3-4	Full CBAM (channel + spatial) on stages 2-3
Skip Connections	None	Every block (identity or 1x1 projection)
Dropout	0.4	0.5
Training	Basic Adam, weak augmentation	AdamW + Mixup + Label Smoothing + TTA
Target Accuracy	~64%	~70%+

Model Specifications

Spec	Value
Total Parameters	2,833,103 (~2.8M)
Stem	704 params
Stage 1 (64ch)	152,192 params
Stage 2 (128ch + CBAM)	529,860 params
Stage 3 (256ch + CBAM)	2,116,292 params
Classifier Head	34,055 params
ONNX Model Size	~11 MB
Input	48x48 grayscale face crop
Output	7 emotion class probabilities
Inference Speed	< 5ms on modern CPUs
Runtime	onnxruntime-web (WASM backend)

Key Improvements in v2

Architecture:

• Residual connections — Skip connections in every block prevent gradient vanishing and allow deeper feature learning
• CBAM attention — Channel attention (avg+max pool -> shared FC -> sigmoid) learns which features matter. Spatial attention (7x7 conv on pooled features) learns where to look in the face
• Stride-based downsampling — Replaces MaxPool with strided convolutions for learnable downsampling
• Deeper classifier — FC(256->128->7) with BatchNorm between layers

Training Techniques:

• Mixup (alpha=0.2) — Blends training images and labels to regularize and smooth decision boundaries
• Label Smoothing (0.1) — Prevents overconfident predictions on noisy FER2013 labels
• CosineAnnealingWarmRestarts — Cyclical LR that escapes local minima and explores the loss landscape
• AdamW (weight_decay=5e-4) — Decoupled weight decay for better generalization
• Stronger augmentation — RandomRotation(15), RandomAffine(shear=5), ColorJitter(0.3), RandomErasing(p=0.25)
• Test-Time Augmentation (5x) — Averages predictions over 5 augmented views at evaluation for +1-2% accuracy
• Gradient clipping (max_norm=2.0) — Stabilizes training with mixup

Training Details

Detail	Value
Dataset	FER2013 (28,709 train / 3,589 test)
Emotions	angry, disgusted, fearful, happy, sad, surprised, neutral
Target Accuracy	68-72% (v1 was 64.4%)
Optimizer	AdamW (lr=0.001, weight_decay=5e-4)
Scheduler	CosineAnnealingWarmRestarts (T_0=15, T_mult=2)
Loss	CrossEntropy + Label Smoothing (0.1) + Class Weights
Regularization	Mixup, Dropout 0.5, Weight Decay, RandomErasing
Epochs	60 (early stopping patience=15)
Training Platform	Google Colab (NVIDIA T4 GPU)

Note: FER2013 has ~65% human inter-rater agreement due to ambiguous labels. State-of-the-art models with 50-100M+ params achieve ~73-76%. Our 2.8M param model targets 68-72% — near human-level accuracy with a model small enough to run in your browser.

Inference Pipeline (Browser)

Webcam Frame
  -> Extract center face region (60% crop at 40% height)
  -> Convert RGBA to grayscale
  -> Resize to 48x48
  -> Normalize (mean=0.5, std=0.5)
  -> ONNX inference via WebAssembly
  -> Softmax -> 7 emotion probabilities
  -> Map to learning state (confused/frustrated/bored/focused/excited/struggling)
  -> Adapt tutor response

This is the beginning of the README. Read the rest at the original. Read the full README ↗