কৃষি ডাক্তার · Krishi Daktar (Bangla Crop Doctor)
A farmer takes or uploads a leaf photo. A computer-vision model running fully inside the phone's browser (no server, works offline) names the likely disease in Bangla + English, shows confidence, the cause and 4 simple tips. Built for the Craft N Code West Bengal hackathon (AgriTech track).
Live demo: https://krishi-daktar-syatem.netlify.app/ (works offline after the first load)
Problem
Small farmers in West Bengal often notice crop disease late and cannot reach an agriculture officer quickly. Wrong or delayed treatment costs yield. Many have poor internet and prefer Bangla.
Approach
| Part | Choice |
|---|
| Crops | Potato, tomato, rice, sugarcane (21 classes: healthy + main diseases) |
| Model | MobileNetV2 (ImageNet) transfer learning, Keras 3 / TensorFlow |
| Training | 2 phases: train new head, then fine-tune top 50 layers |
| Augmentation | flip, rotation, zoom, shift, brightness, contrast, blur, random background replacement (lab images have plain backgrounds) |
| Imbalance | per-sample class weights |
| Export | TensorFlow.js tfjs_graph_model (about 9 MB) |
| App | Plain HTML/CSS/JS, TF.js in browser, PWA (service worker + manifest), Bangla default + English toggle |
| QR camera (computer -> phone) | On a computer, "Take photo" shows a QR code. The phone scans it, takes the photo and uploads it over HTTPS to the free public relay ntfy.sh under a random unguessable topic (ntfy deletes it after about 3 hours); the computer picks it up and the diagnosis runs in the computer's browser. Needs internet on both devices. |
| Hybrid online AI (optional) | Offline: the on-device model above (potato, tomato, rice). Online: the photo is also sent to our own Netlify Function /api/diagnose, which asks a vision AI (Google Gemini). It knows many more crops and diseases (and rice leaf blast / healthy rice) and writes simple Bangla advice. If both agree the app shows "verified"; if they differ it shows the more confident one plus a warning and the other answer. Any online failure silently falls back to the on-device result. The user can switch the online check off. Needs the environment variable GEMINI_API_KEY in Netlify (never in the page). |
| Safety | Below 70% confidence the app says "I am not sure" and sends the farmer to the agriculture officer (Krishi Bibhag). Top-2 predictions are always shown. Disclaimer always visible. |
Classes: potato (early blight, late blight, healthy); tomato (bacterial spot, early blight, late blight, leaf mold, Septoria leaf spot, spider mites, target spot, mosaic virus, yellow leaf curl virus, healthy); rice (bacterial leaf blight, brown spot, leaf smut); sugarcane (mosaic, red rot, rust, yellow leaf, healthy).
Online AI setup
Netlify -> Site configuration -> Environment variables -> add GEMINI_API_KEY from https://aistudio.google.com/apikey (and optionally GEMINI_MODEL, default gemini-2.5-flash), set a spend limit on the key, then redeploy. Photos are sent only when online and the switch is on.
App features (mobile-app style)
Home (live weather + disease-risk hints, Open-Meteo), Scan (on-device model, optional online AI, photo-quality check, listen-aloud and share), Crop guide (offline), AI helper chat (Bangla, voice input, knows the last scan and the weather), My farm (scan history stored on the device, settings, install as app). Online AI answers add severity, spread risk, prevention and a follow-up check question. No account or backend database: history never leaves the device.
Project structure
training/ config.py, prepare_data.py, train.py, evaluate.py, export_tfjs.py, krishi_daktar_colab.ipynb
webapp/ index.html, style.css, app.js, i18n.js, sw.js, manifest.webmanifest, icons/, netlify.toml
webapp/data/ labels.json (class index -> id), diseases.json (Bangla + English cause and tips)
webapp/model/ model.json + weight shards (TF.js)
results/ confusion_matrix.png, per_class_accuracy.png, training_curve.png, metrics.json
Datasets (download yourself, put next to training/)
- PlantVillage (potato, tomato folders) ->
PlantVillage/<class folders>
- Rice leaf disease images (Bacterial leaf blight, Brown spot, Leaf smut) ->
rice_leaf_diseases/<class folders>
How to run
Train (Google Colab recommended, free T4 GPU, about 1 hour):
python training/prepare_data.py builds a train/val/test split (70/15/15, 256 px) and colab_bundle.zip.- Upload
colab_bundle.zip to Google Drive (My Drive), open training/krishi_daktar_colab.ipynb in Colab, run all cells (train, evaluate, export).
- Copy
webapp/model/ and results/ back to this folder.
Run the app locally:
cd webapp
python -m http.server 8080 # open http://localhost:8080
Camera and service worker need localhost or HTTPS.
Deploy: publish the webapp/ folder on Netlify (netlify.toml included).
Evaluation (held-out test split, 2,334 images, 21-class model)
| Metric | Result |
|---|
| Overall top-1 accuracy | 92.6% |
| Top-2 accuracy | 97.6% |
| Share of images with confidence >= 70% | 89.8% |
| Accuracy on those confident predictions | 96.7% |
Per-class accuracy (see results_v3/per_class_accuracy.png, results_v3/confusion_matrix.png):
This is the beginning of the README. Read the rest at the original. Read the full README ↗
