parallax-display

Head-tracking parallax 3D display engine that creates the illusion of depth on a flat screen. Move your head and the screen becomes a window into a virtual 3D space. The view can be driven by webcam eye tracking, the pointer, page scroll, or device tilt, and can switch between them live.

Inspired by Johnny Lee's Wii Remote head tracking project, rebuilt for the browser using MediaPipe face tracking and modern web APIs.

Live demos: ethereal.dev

How It Works

An eye position (where the viewer's eyes are relative to the screen) drives the projection. It comes from a source: the pointer, the page's scroll position, device tilt, or a webcam (see Sources). With the webcam, MediaPipe tracks the viewer's eye position in real time using MediaPipe Face Mesh. The position drives an off-axis (asymmetric) perspective projection that shifts based on where the viewer is relative to the screen. This makes the screen behave like a physical window — objects behind the screen recede as you move, objects in front pop out.

Two rendering backends are supported:

• Three.js — full 3D scenes with geometric depth and off-axis projection
• CSS 3D Transforms — lightweight depth layers for web UIs using perspective and translateZ()

Installation

The package isn't published to npm yet. Install it straight from GitHub:

npm install github:jasondecamp/desktop-vr#semver:^0.3.0

The #semver: range picks the newest matching release tag. npm clones the repo and runs the prepare script, which builds dist/ on install. If your npm version asks you to approve install scripts, allow the build for parallax-display (npm install-scripts approve parallax-display). Without it, the package installs with no built files.

Peer Dependencies

All peer dependencies are optional, depending on what you use:

# Only for webcam tracking (FaceSource, or an engine with no source)
npm install @mediapipe/[email protected]

# Optional (only for parallax-display/three)
npm install three

# Optional (only for parallax-display/react)
npm install react

Library Entry Points

The library is split into three entry points to keep your bundle lean:

parallax-display — Core (no Three.js or React dependency)

Includes the engine, CSS adapter, sources, face tracking, projection math, filters, and the CalibrationPanel (DOM-only UI).

import {
  ParallaxEngine,
  CSSAdapter,
  PointerSource,
  ScrollSource,
  OrientationSource,
  FaceSource,
  CalibrationPanel,
  FaceTracker,
  CoordinateMapper,
  OneEuroFilter,
  EMAFilter,
  computeOffAxisFrustum,
  screenFromViewport,
} from 'parallax-display';

parallax-display/three — Three.js adapter + UI overlays

Includes everything from core, plus the Three.js adapter and all Three.js-dependent UI components (CalibrationOverlay, DiagnosticOverlay, GridRoom).

import {
  // Everything from core, plus:
  ThreeJSAdapter,
  CalibrationOverlay,
  DiagnosticOverlay,
  GridRoom,
} from 'parallax-display/three';

parallax-display/react — React hooks

Includes everything from core, plus React hooks for easy integration.

import {
  // Everything from core, plus:
  useParallaxCSS,
  useParallaxEngine,
} from 'parallax-display/react';

Sources

A source produces eye positions for the engine. Pass one as source, and switch live with engine.setSource():

import { ParallaxEngine, CSSAdapter, PointerSource, FaceSource } from 'parallax-display';

const engine = new ParallaxEngine({
  adapter: new CSSAdapter({ container }),
  source: new PointerSource(),  // no permission prompt
});
await engine.start();

// Later, from a click: opt in to the webcam
cameraButton.onclick = () =>
  engine.setSource(new FaceSource()).catch((err) => showError(err.message));

With no source, the engine uses the webcam (FaceSource) built from the tracking and calibration options, exactly as in v0.2.

Source	Input	Permission	Notes
PointerSource	Mouse or trackpad over the window	None	Touch is ignored (dragging is scrolling)
ScrollSource	Scroll position of an element	None	Pointer is ignored
OrientationSource	Device tilt	iOS motion prompt	start() / setSource() must run in a user gesture
FaceSource	Webcam + MediaPipe	Camera prompt	MediaPipe is loaded on first start

Switching. setSource() starts the new source while the current one keeps driving the view (so the pointer still works while the camera opens), then blends from the current view to the new source over blendMs (default 500, eased), so the view never jumps. If the new source fails to start (permission denied, no sensor), the current one stays and the promise rejects. The previous source is destroyed, which turns the camera off; sources can be started again, so you can pass one back later.

Losing tracking. When a source loses tracking (no face in frame), the view holds where it is and onTrackingLost fires. When tracking returns, the view blends back from where it was held.

PointerSource

The pointer acts as a virtual head at a fixed distance. Moving to the viewport edge leans the eye toward that side of the screen.

const pointer = new PointerSource({
  distance: 0.6,      // meters from the screen (EyePosition.z)
  range: 0.75,        // viewport edge = 75% of the screen's half-width (1.0 puts the vanishing point under the cursor)
  restOnLeave: true,  // ease back to center when the pointer leaves the window
  easeMs: 400,        // duration of that ease, and of easing in on entry
});
pointer.setDistance(0.7); // e.g. a slight dolly during a transition; applied immediately

The engine passes its screen size to the source, so range stays proportional to the viewport after resizes and PPI changes.

This is the beginning of the README. Read the rest at the original. Read the full README ↗