Chalukya AI · ಚಾಲುಕ್ಯ AI

One offline-first AI companion for every tourist journey in Bagalkot district, and one command centre for the people who run it.

Built for AI to Redesign Tourism: Bagalkot 2026 (inter-college AI project competition, 25 Sept 2026), on the World Tourism Day 2026 theme "Digital Agenda and Artificial Intelligence to Redesign Tourism".

Try it: https://chalukya-ai.vercel.app. On a phone, tap Take the 90-second tour on the home screen, or open https://chalukya-ai.vercel.app/?tour=1. It walks through seven screens on its own, with a caption on each, and can be paused, stepped or ended at any time.

It covers all six problem statements in one product, instead of six disconnected demos:

Problem statement	What Chalukya AI does	AI inside
PS 07 Chalukyan sculpture identification	Photograph a sculpture or temple, or point the live lens (AR-style label over the camera view) → name, story, audio narration, location; says "not sure" instead of guessing; site-level answer when unsure between neighbouring temples; Grad-CAM "why"	EfficientNet-B0 transfer learning, ONNX on the phone (offline)
PS 12 Multilingual voice assistant	Speak or type in Kannada, Hindi, English or romanised text; answers in the same language, read aloud, with live data cards and sources	Hybrid intent model (char n-grams + curated lexicon + multilingual-e5), tool-using answer composer grounded in a verified KB
PS 18 Hotel recommendation	Budget, distance to your itinerary, must-have amenities, priorities → ranked list with "why this"	Multi-criteria hybrid ranking + Bayesian rating + inclusive-growth boost + diversity (MMR) re-rank
PS 19 Local restaurant recommendation	Veg / non-veg, budget for two, cuisine, "jolada rotti meals", open now	Same explainable ranker; diet inferred from names where obvious
PS 25 Tourist safety alerts	Hold-to-SOS with location, geofenced caution zones, missed check-in alarm, live heat index, CCTV people counting (privacy-blurred), IoT anomaly detection → command centre in real time	Person detector (Faster R-CNN), robust z-score anomaly detection, forecast-driven overcrowding alerts
PS 21 Smart parking prediction	Live free slots per lot + predicted free slots on arrival	LightGBM 30/60/120-min forecasts; a 120-tree version runs on the phone
Heritage OCR + translation	Photograph a Kannada/Hindi/English information board → which monument it is, its story in your language (offline), full board translated into 6 languages	Tesseract LSTM on the phone; KB grounding by heading-line names; Meta NLLB-200 (int8, glossary-protected names, streamed) on the server
Department insights	Reviews in any language → sentiment + complaint topic per site; what tourists ask, in which language, about which place	Review model (char n-grams + polarity lexicon on the phone, multilingual-e5 on the server); anonymous usage counts
Open innovation	Crowd-aware trip planner; 7-day water/waste/staff planning; Local Business Portal; offline PWA	LightGBM footfall forecasting with conformal intervals; exhaustive route optimisation

Results (honest numbers)

Model	Test design	Result	Baseline
Sculpture recognition (25 classes, 1,960 CC-licensed photos from 202 photographers)	photographer-held-out test set (344 photos)	85.8 % top-1, 91.9 % top-3, 98.0 % correct site, macro-F1 0.83	—
… with the "not sure" rule	same	answers 79 % of photos at 95.6 % accuracy; rejects 82 % of non-Chalukyan photos (n = 33)	—
Footfall forecast (6 sites)	time-based hold-out, Jan–Aug 2026	WAPE 13.1 %; festival days 14.9 %	same-day-last-year 31.1 % (42.7 % on festivals)
Forecast intervals	same	79.2 % coverage for a nominal 80 % band (conformal)	—
Parking (6 lots)	winter peak season hold-out	±3.5 slots at 2 h ahead; "lot full" alerts 81 % precise	"stays the same": ±7.9
Intent understanding (17 intents, 3 languages + romanised)	unseen phrasings / fresh test set	server hybrid 95.8 % / 88.6 %; phone-only 91.2 % / 78.6 %	char model alone 51.7 % / 68.6 %
Review sentiment (3 classes, 4 scripts/styles)	30 separately written test reviews	93.3 % (multilingual-e5); complaint topic F1 0.98	char model alone 73.3 %
Heritage board reader	4 real ASI boards (Commons photos)	4/4 matched to the right monument or site; translation 3.9× faster with int8, first sentence in ~6 s	—

The phone's review lexicon scores 96.7 %, but the same person wrote its word lists and the test set, so the e5 number is the fair one.

Lighthouse, measured on the live site on 19 Sept 2026 with a simulated mid-range phone on slow 4G (home page): Performance 83–87, Accessibility 100, Best Practices 100, SEO 100. Accessibility is also 100 on Ask, Plan, Scan, Safety, Command and How.

Full reports: ml/vision/out/efficientnet_b0/, ml/forecast/out/, ml/assistant/out/.

What is real and what is simulated

Real: 1,960 monument photos (Wikimedia Commons, CC licences, credits in ml/vision/data/metadata.csv); ASI annual footfall (Badami 4,44,542 · Pattadakal 3,24,615 · Durga temple Aihole 2,13,901); OpenStreetMap places, hospitals and road network (OSRM); facts cross-checked against the District Tourism Development Committee's book Sounds and Emotions (2022, paraphrased); the District Administration's tourism pages (bagalkot.nic.in) and tourism office contact; Open-Meteo live weather.

AI-enhanced (and labelled): the home-screen photos are real Commons photographs relit and upscaled to 4K with Nano Banana 2; the architecture is unchanged. 4K masters are in docs/brand/hero-4k/. The logo (Recraft vector, chosen by the author) is in docs/brand/logo/ as SVG, 2048 px and 4K PNG.

Simulated (and labelled in the app): the daily/hourly shape of footfall (calibrated so the annual totals match ASI exactly), parking sensor streams, business prices/ratings/hours. Every pipeline retrains unchanged on real ticket-counter or sensor data.

Run it

# one click on Windows
START_DEMO.bat
# or manually
.venv\Scripts\python -m server        # web app + APIs on :8300 (https :8443 for phones)
.venv\Scripts\python iot\simulator.py # IoT stream (or flash iot/parking_node to an ESP32)

Open http://localhost:8300 (tourist app) and http://localhost:8300/command (district command centre). Phones on the same Wi-Fi: use the https address the server prints (accept the certificate once; needed for mic and GPS).

Without the server, cd web && npm run dev still runs the whole app: vision, intent, recommenders, forecasts, planner and parking model all run on the device.

Repository map

ml/vision/      dataset builder, photographer-grouped split, training, open-set calibration, Grad-CAM (server)
ml/forecast/    calendar features, calibrated simulator, footfall + parking models (+ phone export)
ml/assistant/   trilingual intent data, lexicon, training + comparison, fresh test set
ml/kb/          curated knowledge base (en / ಕನ್ನಡ / हिन्दी) with sources
ml/data/        OpenStreetMap businesses, OSRM distances, photos with credits, CCTV samples
web/            React + TypeScript PWA (tourist app + command centre)
server/         FastAPI: WebSocket alerts, IoT ingest + anomaly detection, vision, intent, people counting, TTS
iot/            ESP32 firmware (parking, safety) and a simulator
docs/           pitch, demo script, judges' Q&A, registration answers

Credits

Photos: Wikimedia Commons contributors (see ml/vision/data/metadata.csv and web/src/data/photo_credits.json). Map data © OpenStreetMap contributors. Weather: Open-Meteo. Footfall totals: Archaeological Survey of India. Reference: District Tourism Development Committee, Bagalkote, Sounds and Emotions (2022). Kannada and Hindi text should be reviewed by a native speaker before public release.