AR Menu — proof of concept

Scan the QR code with a phone, pick a dish and tap View on your table. The dish appears on your table at its real size. Walk around it and lean in.

Dish	Real size
Chocolate ring cake	26 cm across, 11 cm tall, on a 32 cm plate
Tiramisu	a layered slice in a 16 cm tray, 7 cm tall (AI-generated model)
Pumpkin gnocchi	23 cm bowl, 7 cm tall to the top of the food (AI-generated from photos of the real dish)

Each dish has its own link: …/ar-menu-poc/#cake, …/ar-menu-poc/#tiramisu, …/ar-menu-poc/#gnocchi.

Live: https://pmtchakanyuka-star.github.io/ar-menu-poc/

How it works

Piece	What it is
index.html	The menu page. One tab per dish (the DISHES list in the script). It uses Google's <model-viewer> for the 3D preview and the AR hand-off. ar-scale="fixed" locks each dish to its real size.
models/*.glb	glTF 2.0 models for the web preview and Android: WebXR in Chrome, with Scene Viewer as the fallback.
models/*.usdz	USDZ models for iPhone and iPad (AR Quick Look).
models/*-thumb.webp	Menu tab pictures: renders of the models.
tools/calib/detect_edges.py	Finds the plate rim, the edge of the glaze pool and the cake's top outline in the reference photo.
tools/calib/fit_camera.py	Fits a camera and the cake's profile to those edges, plus a few hand-measured points on the hole and base, then scales the result so the base is 26 cm.
tools/build_cake.py	Builds the shape and projects the photo onto it as texture. Writes both model files.
tools/build_ai_model.py	Turns an AI-generated 3D model (Higgsfield → Tripo H3.1, downloaded into the git-ignored tools/ai/) into a menu model: real size, welded and decimated to 130k triangles, a fresh UV atlas with the full-detail texture baked onto it, matte food. Writes both model files. Used for the gnocchi and the tiramisu.
media/hero.*	The looping hero clip (Higgsfield Seedance from a photo of the gnocchi at the café, WebM + MP4, ~0.2 MB) and its poster. It starts only after the first dish has loaded, never with reduced motion or data saver, and has a pause button.
tools/build_tiramisu.py	The earlier, photo-built tiramisu (no longer used on the menu).
tools/build_gnocchi.py	The earlier gnocchi: fits a camera to the overhead photo and textures a height field with it (no longer used on the menu).
tools/model_io.py	The glTF and USDZ writers both builders share.
tools/preview.html	Dev-only full-window render of one model (?src=tiramisu).
tools/make_qr.py	Writes qr.png / qr.svg.

The model is made from one photo

1. Shape. A pinhole camera and the cake's profile are fitted to the photo. The profile runs from the pool of glaze in the hole, up the hole wall, over the rounded crest and down the flared sides to the base. The fitted edges land within a few pixels of the photo. The only real-world number supplied is the base width (26 cm, a standard ring pan); everything else comes from the photo. The glaze pool's wavy outline on the plate is back-projected from the photo too.
2. Texture. The photo is projected onto that shape through the fitted camera, so everything the camera saw head-on keeps its real pixels. One photo can't see everything (the back, the near wall of the hole), so those parts are filled with randomly chosen patches of the seen surface at the same height, with crossfades and no mirroring. First the photo's lighting is evened out around the cake, so patches match and the AR viewer's lights do the shading. A bump map and a roughness map come from the same pixels: glossy glaze pools, satin sprinkles.
3. Export. Both files are in metres and Y-up, with the origin under the plate, so AR puts the cake straight onto the table.

Cake	26 cm across the base, 11 cm tall (31.6 cm plate)
Triangles	~101k
Textures	colour 2048×1024, bump 2048×1024, roughness 1024×512 (JPEG)
Files	cake.glb 4.4 MB · cake.usdz 3.3 MB

Both files are validated. The Khronos glTF Validator reports 0 errors and 0 warnings, and all 28 OpenUSD validators pass for the USDZ, including the usdz packaging rules.

The tiramisu is made from five photos

1. Shape. A slice with a gently domed top, soft rounded edges and cream sides that bulge and slump a little, on a black thermoformed tray with ribbed walls and a rolled rim, plus a three-leaf mint sprig. There is no real-world measurement in the photos, so the tray is the yardstick: it is about ten times as wide as it is deep, and the slice spans about 70% of its width. The sizes are constants at the top of tools/build_tiramisu.py.
2. Texture. The top is the cocoa from the straight-down photo. The sides are the slice's front face from two photos, warped flat by the four corners of the face and wrapped round, so every pixel is a real one. Where the top turns into the side, the cocoa gives way to the cream along a ragged line, as dusted cocoa does. The lighting is evened out first, and the bump and roughness maps come from the same pixels: matte cocoa, moist cream.
3. Export. Like the cake: metres, Y-up, origin under the tray.

Slice	11 × 9 cm, 4 cm tall (16 cm tray)
Triangles	~113k
Files	tiramisu.glb 4.2 MB · tiramisu.usdz 3.1 MB

Both files pass the same checks as the cake's: 0 glTF Validator errors or warnings, and all 28 OpenUSD validators.

This is the beginning of the README. Read the rest at the original. Read the full README ↗