omg-audio

Live demo: https://justgoscha.github.io/omg-audio/web/ — headphones on. Desktop: WASD + mouse, Space throws a whistling, exploding ball. Android Chrome: on-screen joystick, turn your body to look around (device orientation drives the binaural rendering).

The engine today is v5.5 — rays from the listener, every connection reaching each source where it was when it emitted, the traced response rendered as traced: ENGINE_V3.md (the tracer), ENGINE_V4.md (the response rendered as traced), ENGINE_V5_5.md (what v5.5 changed), measured in docs/runs/, tested per TESTS.md, open items in OPEN.md. Parts of the overview below describe the earlier engine (portals, image sources) that the offline renderer still uses.

A real-time path-traced sound propagation engine in Rust, compiled to both native (reference build) and wasm (Web Worker + AudioWorklet). One scene — a small city square with a house playing music, a great hall with a narrator, a club with four synchronized speakers behind an entrance vestibule, a two-storey old house, a colonnade and a kiosk — and the sound field is computed, not authored:

• Image-source early reflections (order 3) + stochastic path tracer (4096 rays, 3 frequency bands) measuring per-room RT60 and late level.
• Portals: sources in other rooms render as virtual sources at doorways (BFS over the door graph), each crossing priced by knife-edge diffraction at the jamb the path actually bends around — free on the sight line, bass-only around the corner; equal-power blended near thresholds. Coupled rooms carry their own reverb through the doorway as a directional wet emitter — you hear the hall being a hall from the corridor.
• Wall transmission: straight rays cross wall segments with mass-law attenuation (amplitude ∝ reference/thickness) and a fitted one-pole lowpass — European brick and concrete, not drywall. The club reads as oomp-oomp from the street and opens up as you walk in.
• Aperture radiation: doors and windows re-radiate what's inside — including order-2 reflections from the source room — so a window is audible off-axis, not just on the sight line.
• Diffraction as an occlusion floor: losing sight of a source (or of the window/door it radiates from) never cuts it — for every exit emitter the simulation finds the best bent path around the blocking geometry (single and double corners, over the roof) and occlusion floors at its Kurze–Anderson knife-edge loss. The Fresnel number of each path's detour decides, per band, how much survives: bass wraps around buildings, treble shadows. Dry sound, early reflections and coupled reverb all hand off to the bend together — a regression test literally walks through a building's shadow and asserts no level jump. Plus facade reflections outdoors.
• Binaural output: order-2 ambisonics decoded through 20 virtual speakers (dodecahedron) × measured MIT KEMAR HRIRs, plus point rendering — the strongest N paths per source get their own nearest-HRIR convolution from a 710-direction grid, N adapting to measured CPU load.
• Ear adaptation: an AGC protects against ultra-loud content (club PA, explosions) with fast clamp and ~30 s recovery — it never boosts quiet. On top of it, hearing fatigue (temporary threshold shift): seconds of demand ~10 dB over the target build up a muffle — a lowpass sweeping 18 kHz → 1.4 kHz with exposure depth — that releases over ~25 s, the dulled-ears feeling after stepping away from the speakers. The HUD shows muffled N% while it holds.
• Doppler by construction: tap delays glide on motion; path identity changes crossfade. Room transitions cannot click or chirp.
• The ambient dome: ambience and rain are not listener-glued beds — they are the outdoor field, an audio skybox sampled by rays. Every tick a deterministic fan of 512 rays traces from the listener against a real mesh of the scene (walls with true aperture holes, slabs, roofs, transmissive glass panes, swinging door leaves); rays that escape to the sky deliver the dome's sound from their departure direction, attenuated by whatever they bounced off or passed through. Standing outside, the whole dome arrives; indoors, the openings localize the world outside; a room with no direct opening receives it through the rooms that do — after threading two doorways, emergently. What rays cannot carry — the through-shell seep — comes from a Sabine power balance over the room-coupling graph (shared walls, slabs, roofs, exterior surfaces, live apertures, all derived from the scene). No per-room constants anywhere.

Demos

Every engine version re-renders the same scripted 98 s walk (living room → corridor → great hall → outside → entrance → club) as a top-down schematic video with binaural audio — the whole history is on the walkthroughs release, headphones on:

version	what it demonstrates
v12 – huygens-doorways	apertures re-radiate as Huygens sources; doorway wet pays Lambert obliquity — a real on-axis peak walking past an open door
v11 – mesh-dome-cathedral	ambient dome, mesh-emergent knife-edge occlusion, cathedral + bunker
v10 – club-transmission	mass-law wall transmission — oomp-oomp from the street
v8 – point-hrtf	per-path nearest-HRIR convolution for the strongest paths
v7 – hrtf-binaural	measured KEMAR HRIR binaural decode
v6 / v5 – coupled reverb	coupled rooms carry their own reverb through doorways
two-sources / voice / music	the earliest renders

New versions join with gh release upload walkthroughs walkthrough-vN-<name>.mp4 after the render recipe below.

Hear it

Web (what the live demo runs):

sh tools/build_web.sh          # cargo build wasm (+simd128) + stage into web/
python3 tools/serve.py         # http://localhost:8000/web/
node tools/web_smoke.mjs       # headless pipeline test
node tools/env_probe.mjs       # ambience/rain continuity through doors + swings
node tools/bench_web.mjs       # realtime-factor benchmark (point budgets)

Native:

cargo run --release              # live playback, Ctrl+C to quit
cargo run --release -- --render demo.wav --secs 12   # offline render + level report
cargo run --release -- --input assets/aria48.wav --render out.wav

# scripted walkthrough with fixed sources + 2D schematic video
cargo run --release -- --walkthrough --render walk.wav --json walk.json \
    [--music m.wav --voice v.wav --club c.wav]
python3 tools/render_viz.py walk.json frames 30
ffmpeg -framerate 30 -i frames/%05d.png -i walk.wav \
    -c:v libx264 -pix_fmt yuv420p -c:a aac -shortest walkthrough.mp4

Env knobs (native): OMG_POINT_TAPS=n point-render budget, OMG_MUTE_TAPS / OMG_MUTE_OWN / OMG_MUTE_REMOTE isolate signal paths.

Godot 4 (GDExtension, tested against 4.7):

sh tools/build_godot.sh        # cargo build the extension + stage into godot/bin/
/Applications/Godot.app/Contents/MacOS/Godot --headless --path godot -s test.gd   # smoke test
/Applications/Godot.app/Contents/MacOS/Godot --path godot                          # playable demo

The extension is omg-audio (crates/omg-audio) as Godot nodes — nothing demo-specific:

• OmgWorld — the engine: omg-audio's threaded::Engine (simulation thread; v3 on the GPU with use_gpu, CPU otherwise). rebuild_geometry() collects the level's triangles (MeshInstance3D surfaces) into an omg-audio Geometry; an output thread renders binaural audio ahead into a ring that Godot's audio thread pulls through OmgOutputStream — the game thread never runs DSP or waits on the engine. Also: load_clip(path, options) (Ogg Vorbis/FLAC/Opus, FLAC, WAV decoded in Rust in the background, with the web demo's shaping options), whisper() (near field at one ear), ambience bed + rain routed by the frame's sky (ambience_file, rain_bank_file, rain_intensity), ground_plane, quality, stats(), and the web worklet's load governor over omg-audio's mix-wide budgets.
• OmgGeometry — tags everything below it with an acoustic material (material = "brick", see OmgWorld.material_presets(); "none" opts out); as_boxes makes its meshes solid boxes. Simplest of all: give a visual Material an omg_material meta once and every mesh using it is acoustic — the texture brings its sound.
• OmgSource — a looping file and/or one-shots (play_clip), at a real SPL (level_db at 1 m).
• OmgListener — the ears (falls back to the current camera).
• OmgDoor — a leaf or pane filling an opening (an omg-audio object box), priced by openness.

godot/demo.gd + godot/demo/ is the web demo's town at night, generated in GDScript (same rooms, doors, windows and landmarks as web/main.js / the walkthrough scene; furniture from assets/demo/layout.json when present), dressed with CC0 textures (assets/textures, Poly Haven) whose materials declare their acoustics (--audit prints every texture → acoustic pairing), and the web demo's sound design (room loops, footsteps, owls, bells, whispers, cars, balls, rain). It renders a cheap pixel look by default (3D at 360 px high, width from the window's aspect, nearest upscale, no anti-aliasing); G switches to the fine look (full resolution, SSAO/SSIL, volumetric fog), [ and ] change the height. Controls: click to capture / throw, WASD, Shift run, Space jump, C crouch, F fly, E door, B bells, V whisper, R rain, L labels, Q sound quality, H hud. -- --print-fps measures ~10 s of frame rate, -- --shots=<dir> renders a tour of viewpoints to PNGs, -- --gpu traces on the GPU.

Architecture — the two clocks

The load-bearing decision: the simulation clock and the audio clock never share state except through ParamBlock.

┌─────────────────────────────────┐   ParamBlock   ┌──────────────────────────────────┐
│ SIMULATION  (20 Hz)             │  ───────────▶  │ AUDIO  (per-sample)              │
│ Web Worker / native thread     │                │ AudioWorklet / cpal callback     │
│                                 │  taps: key,    │                                  │
│ omg-core + omg-scene            │  delay, dir,   │ omg-dsp                          │
│  · image sources (ISM ≤3)      │  band gains    │  · fractional delays (Doppler)   │
│  · stochastic tracer → RT60    │                │  · per-tap shelf EQ + wall LP    │
│  · portals, transmission,      │  reverb:       │  · point HRIR convolution        │
│    apertures, diffraction      │  rt60[3],      │  · order-2 ambisonic bus         │
│  · EMA temporal accumulation   │  level[3],     │  · 8-line FDN (traced RT60) ×2   │
│    (kills MC flutter)          │  remote wet    │  · 20-speaker KEMAR decode + AGC │
└─────────────────────────────────┘                └──────────────────────────────────┘

• Every parameter crossing the boundary goes through a one-pole smoother — simulation updates can never click, and moving delays are the Doppler.
• Taps carry stable path-identity keys: same key ⇒ glide (motion), new key ⇒ crossfade from silence. Delay is never slid between two different paths — that would be a pitch chirp.
• Web transport is flat ParamBlocks over postMessage (~4 KB @ 20 Hz) — no SharedArrayBuffer, no COOP/COEP headers, so it hosts on GitHub Pages.
• Head orientation (mouse look, device orientation, camera face tracking) is a fast path straight into the DSP: a full yaw/pitch/roll SH rotation (exact block-diagonal 3×3 + 5×5 on the order-2 bus) at the decode stage
  • per-block nearest-HRIR re-selection for point taps.
• The 🎥 button in the demo enables camera face tracking (MediaPipe FaceLandmarker, fetched on demand): your real head movements — small turns, tilts, even leaning — drive the engine's head while the view stays put. Turn your head and hear the world hold still. recenter re-zeros on your current sitting pose.
• The late field is an FDN driven by traced per-band RT60, not a convolved impulse response — parameters morph artifact-free in real time.

Rendering: bus + points

Two tiers, chosen per tap by measured loudness:

1. Point rendering — the strongest N taps per source (direct sound, prominent early reflections) each get their own convolution with the nearest of 710 measured KEMAR HRIRs: full localization sharpness where localization actually happens. Cost is linear in N.
2. Ambisonic bus — everything else (dense reflections, reverb tails, the ambience bed) sums onto one shared order-2 bus, decoded once through 20 virtual speakers × KEMAR HRIRs. Fixed cost regardless of tap count, and the physically right tool for diffuse content, which has no single arrival direction to sharpen. Spatial resolution is bounded by the ambisonic order (~±30° blur at order 2), not the speaker count — 20 speakers already saturates order 2, which is why the path to sharpness is tier 1, not more speakers.

N is not hardcoded: the AudioWorklet measures its own render load and walks the budget between 8 and 32 with hysteresis — a throttled CPU (Low Power Mode, phones) settles low, a desktop climbs to the cap. The HUD shows the current value (hrtf ×N). Measured on an M1 at the worst-case scene position (club/entrance doorway blend, ~446 live taps, SIMD kernel):

point budget / source	realtime factor
0 (bus only)	3.0×
8	2.5×
24	2.1×
32	1.9×
64	1.3×

Rendering every tap this way (up to 160/source × 6 sources) would be ~10× over budget — that, and only that, is why the bus tier exists. The convolution kernel is written to vectorize (NEON / wasm simd128): pre-reversed HRIRs turn the ring convolution into contiguous dot products with explicit 4-lane accumulation.

Crates

crate	role	wasm?
omg-core	vec/rng, materials, shoebox + triangle-mesh geometry (binned-SAH BVH, budgeted diffraction-edge extraction), ISM, path tracer, ParamBlock. Zero deps.	yes
omg-dsp	delays, smoothers, EQ, FDN, ambisonics, HRTF (bus + point), renderer, output stage. Alloc-free hot path.	yes
omg-scene	the world: rooms, doors, windows, materials/thickness, portal graph, transmission/aperture/diffraction routing, WorldSim	yes
omg-web	wasm C-ABI exports (sim_* for the Worker, eng_* for the Worklet), no wasm-bindgen	is the wasm
omg-app	native binary: cpal live output, offline render, walkthrough scripting, JSON export for the video tool	native
omg-godot	GDExtension (godot-rust): omg-audio as Godot 4 nodes (OmgWorld, OmgGeometry, OmgSource, OmgListener, OmgDoor), KEMAR HRIRs baked in	native

Deliberate approximations (and what full fidelity would need)

Everything here is a measured trade against the real-time budget, not a guess — tools/bench_web.mjs is the receipt:

• 3 frequency bands (<250 Hz, 250–2500, >2500). Wall/air filtering is fitted continuously from the bands (one-pole lowpass through the band gains), so nothing sounds stepped; production engines use 4–9 bands.
• The demo scene authors rectangles, but the core is no longer limited to them: omg_core::mesh is an arbitrary triangle mesh with a binned-SAH BVH — the same stochastic tracer runs over it (unit-tested to match the analytic shoebox on a box), segments query per-surface transmission crossings, and diffraction edges are auto-extracted with an explicit importance budget (dense meshes degrade by dropping short/ shallow creases first, never by breaking). Measured on M1 (mesh_bench, 2 520-tri city block): BVH build 2 ms, 2 M rays/s, full trace 79 ms at the 4096-ray desktop tier / 19 ms at the 1024-ray mobile tier — variance, not bias, is what degrades on cheaper tiers (EMA accumulation smooths it). On top of it, omg_core::paths finds propagation paths on a raw mesh with ZERO authoring — no rooms, no doors, no portals: wall thickness is emergent (entry/exit crossings paired for the mass law), and door jambs, building corners and roof lines are all just auto-extracted edges searched for single- and double-bend paths (166 µs per source-listener query at the default budget). The demo's OCCLUSION now runs on this: the old hand-built corner list, corner-visibility matrix and multi-roof rubber band are deleted — occlusion floors come from AutoPaths over the same world mesh the ambient dome traces (bound-ranked single bends, branch-and- bound double bends, an over-the-silhouette hull path for any number of buildings, bent energy averaged over each aperture's real extent, and per-cell caching with a fixed refresh budget). In-room acoustics remain per-room shoeboxes (which these rooms are); the directional late field and the wgpu compute port are the remaining milestone.
• Diffraction is knife-edge Kurze–Anderson (Fresnel-number insertion loss per edge, "rubber band" multi-edge construction) over corner, roof and door-jamb edges — the dominant behavior, but not full UTD wedge coefficients (interior wedge angle, reflection-boundary terms). And as an occlusion floor it preserves level and spectrum continuity, not arrival direction: deep in a shadow, bent energy still arrives from its emitter's direction rather than visibly from the corner or roof line (near the boundary, where localization is sharpest, the two coincide).
• Order-2 bus for the diffuse tier — see the rendering section; the sharp tier bypasses it entirely.
• Nearest-HRIR selection (with 10 ms crossfades) rather than interpolation; the 710-point grid keeps neighbor error small.
• One FDN per source room pair (own + coupled), not a full acoustic radiance transfer.

License & attribution

Code: Apache-2.0. Acoustic data and demo media:

• MIT KEMAR HRTF measurements (Gardner & Martin, MIT Media Lab) — free with attribution.
• Kimiko Ishizaka, Open Goldberg Variations — CC0.
• LibriVox Alice's Adventures in Wonderland — public domain.
• Night ambience: "nature_ambient_night_5-min" by edmondsr — CC0.
• Cathedral flute: "Natural Native Flute with Lush Reverb" by CVLTIV8R, performed by RJ Roush — CC0.
• Bunker radio: "An old radio playing in a Sri Lankan sewing factory" by florianreichelt — CC0.
• Car motors: "car_idle_loop" by AndrewAlexander, "Bus Motor/Engine Sound Loop" by qubodup, "Diesel Generator 03" by KVV_Audio, "Motor Loop" by soundjoao — all CC0, loopified via tools/make_motors.py.
• Club track, projectile FX: synthesized by this repo's tools (CC0).
• Rain splat bank: 22 hits sliced from CC0 freesound recordings (sources listed in tools/make_drops.py).

assets/*.wav are gitignored; regenerate via tools/ or use the shipped .ogg versions (what the web demo loads).