Year of Frames — image → sound → music

Turns pictures into sound, sound back into pictures, and either one into playable music. It began with a year of timelapse photographs and grew from there: any recording can be read as a picture, pictures hidden in sound can be found, motion in a video can be heard, and music from clay tablets and carved stone can be played again. Each tool is standalone; chain them or stop anywhere.

photo ─────┐                          ┌──► picture again (round trip)
recording ─┼──► spectrogram audio ────┼──► notes / MIDI, any tuning or instrument
video ─────┘    (motion only)         ├──► recurrence plots (structure + anomalies)
                                      └──► audio diffs (what changed)

Try it in your browser

unixabg.github.io/imgaudio runs the same imgaudio.py, notate.py and lenses in the browser, on phones too, with nothing to install and nothing uploaded. Start from a photo or from any recording; choose a lens, a tuning (including the seven Babylonian ones) and an instrument; hear the result, see the picture read back out of it, and save the WAV and MIDI. See The web page for building and deploying it.

Setup

sudo apt install git python3-venv imagemagick ffmpeg sox

git clone https://github.com/unixabg/imgaudio.git ~/imgaudio
cd ~/imgaudio
python3 -m venv .venv
source .venv/bin/activate
pip install numpy scipy Pillow matplotlib

Activate with source .venv/bin/activate each session, or call the venv python directly: ~/imgaudio/.venv/bin/python3 imgaudio.py .... To update later: cd ~/imgaudio && git pull.

Layout

imgaudio.py              image ↔ audio, with lens support
notate.py                audio → musical notes + MIDI, in any tuning and instrument
motion.py                video → sound of only what moves
anomaly.py               recurrence plots + anomaly ranking
render.py                runs the pipeline from a TOML recipe
lenses/                  edges, fractal, phyllotaxis, reveal, spectral, _example
recipes/                 example.toml, recordings.toml, sweep.toml
imgaudio-batch.sh        encode every frame
imgaudio-diff-batch.sh   per-frame diffs + change_log.txt
lens_recursion.sh        feed an image through lenses N times
knobs.sh                 sweep anomaly.py settings on one file
examples/                demos, test clips, and ancient music (see Examples)
web/                     the browser version (see The web page)
VERSION                  release number shown on the web page
frames/  out/  renders/  data (gitignored)

From a photo

python3 imgaudio.py --auto-prep encode photo.jpg tmp.wav
python3 notate.py notes tmp.wav melody.wav --midi melody.mid

~5 seconds. melody.wav plays immediately; melody.mid opens in a DAW. Add --voice lyre or --voice piano for another instrument (see Instruments).

From a recording

Any WAV works, whether or not it came from this pipeline.

python3 imgaudio.py --rows 400 --cols 800 decode mystery.wav look.png

No lens, no other flags — --lens raw is the default and shows the honest spectrogram. If the audio was produced by imgaudio.py encode, the picture reappears. If it wasn't, you get a real image of the sound's structure. --base is a notate.py flag and has no effect here.

Trim long files first, or the image comes out absurdly wide: sox mystery.wav short.wav trim 0 30. Twenty to sixty seconds reads well.

Two failure modes: content squashed at the bottom means the audio sits below the default 80–8000 Hz band, so try --f-lo 40 --f-hi 4000; a flat gray field means the audio is too self-similar at that resolution, so drop to --rows 200 --cols 400.

Reading it with rulers

--annotate also writes look.annotated.png: the same picture, enlarged, with seconds along the bottom, Hz on the left and note names on the right (every C, plus A4). With --palette octave it adds a key of the twelve note colours.

python3 imgaudio.py --rows 300 --cols 600 --annotate decode mystery.wav look.png

look.png itself is unchanged, so it still encodes back into sound; the annotated copy is only for reading. Read a note's pitch from the middle of its line; the annotated copy prints that rule under the picture, since telling readers how to read was one of the biggest helps in the blind tests. Below about 300 Hz the lines widen into bands (the decoder's pitch steps are about 21.5 Hz apart at 22.05 kHz), so low notes are only accurate to about a semitone.

In two blind rounds, readers (language models, two per condition, reading by eye) judged the same 12 clips. With the rulers they named the first note exactly 24 and 17 times out of 24, and within a semitone 24 and 23 times. Given only a written description of the axes: 19 and 14 exactly. With no axis information: 0 and 9, and the misses were all the same error, every pitch about eight semitones low from a wrong guess at the range. So the rulers help most by keeping the scale attached to the picture, and the remaining misses are the last semitone of low notes, where the band is wider than the interval. Timing was read well in every condition.

Reading it as text

--annotate-text writes look.annotated.txt: the notes in the sound as plain text, one line each, with when it starts, how long it is heard, its name, its pitch in Hz, and how many cents it sits from the equal-tempered note (so the base-60 tunings show as they are). A line above the list says what kind of sound it is: how notes begin and fade, how bright the tone is, how much noise lies between the notes, and whether the pitch wavers.

python3 imgaudio.py --annotate-text decode mystery.wav look.png

start s   lasts s  note   Hz        cents
   0.00     0.95  A4        440.0    +0 ±2
   0.49     1.71  C4        261.6    +0 ±2

It reads the sound itself with longer windows than the picture uses, so its pitches are good to a few cents even low down, where the picture's bands are a semitone wide. On 63 synthesized clips across all seven instruments (melodies, fifths and chords) it found 94% of the notes, named the first note right in 62 of 63, and its pitches were within 3 cents of the truth 95% of the time. Extra notes it lists are mostly overtones or a ringing note counted twice. It is weakest on the piano's low chords, on two notes an octave apart that start together (it may list only the lower), and on sounds that are not notes at all, which its noise figure warns about.

In a blind round, fresh readers (language models, two per condition) judged 12 new clips. They named the first note exactly 24 of 24 times from the text, 20 of 24 from the picture with rulers, and 24 of 24 when given both. All of the first four notes right: 22, 16 and 23 of 24. The two text misses were one faint false note a semitone below a real one (since removed). Readers given both mostly read only the text. The text cost each reader about 1,500 fewer tokens per clip than the picture, and took seconds rather than minutes to read.

The motion summaries were tested the same way: 12 generated scenes (0 to 2 swaying things anywhere, camera still, shaken or bumped), text only, three fresh readers, key kept off disk. All three gave identical answers. Camera state 12 of 12 (every bump time within 0.1 s), how many things 11 of 12, things found 12 of 13, left-to-right third 12 of 12, swings per second 12 of 12 (within 0.02), top-to-bottom third 10 of 12. The one miss was a shaken scene with two swayers where euler found one and rhythm the other; readers kept only one. About 700 tokens of text per clip.

Two changes followed. --motion-lens euler,rhythm reads the same frames with both lenses and ends with a [both] section: each thing either lens found, whether both saw it or only one, and pieces of one thing joined. And when the camera shook, euler now says whether a mover keeps a steady rhythm the camera's own motion does not have, which shows it is a real mover rather than camera motion left behind. (A reader asked why they had dropped the second thing pointed to exactly that doubt.) On 12 new scenes, generated after the changes and read by two more fresh readers: camera state 12 of 12 (bump times within 0.1 s), how many things 12 of 12, things found 13 of 13, left-to-right 13 of 13, swings per second 13 of 13, top-to-bottom 10 of 13; all three misses lay within 2% of the line between two thirds.

python3 motion.py read wind.mp4 both.wav --motion-lens euler,rhythm

Hidden in plain sight

The lenses above read what visibly moves. hidden looks for what does not show: a patch of the picture whose brightness, colour, position or texture rises and falls more regularly than chance, however faintly. It searches six channels a generic camera records (brightness; blue-yellow and red-green apart from brightness; sideways and up-down shifts far below a pixel; texture), for every patch and every rate, and gives each finding the chance it is luck across the whole search, and its size. It sorts what it finds: too small to see, or large enough to see; echoes at multiples of a rhythm are folded in; rhythms the whole picture shares are the camera's or the light's; rhythms mirrored about the camera's uneven frame pattern are probably the camera's. In night mode it says so: infrared is itself light beyond human sight.

listen is a tuning dial for the same thing, for the ear. Every patch is a voice (high pitch = top of the picture, left ear = left), and a patch sounds only when much of its own change repeats at the tuned rate. Wind spreads over all rates and stays silent; a regular flicker rings. --sweep plays the clip once per rate, sped up, each pass announced by beeps (1 beep = 0.5 a second, then 0.7, 1, 1.4, 2, 2.8). The dials: --quiet (how regular a patch must be to sound; lower hears fainter things and more noise), --width-tune, --gain, --speed, and --channel (any of the six). It writes a report of what sounded at each setting and where.

python3 motion.py read clip.mp4 hid.wav --motion-lens hidden
python3 motion.py listen clip.mp4 dial.wav --sweep
python3 motion.py listen clip.mp4 dial.wav --tune 1.7 --channel shift-up-down --quiet 0.15

To see which faint rhythms keep coming back, read a folder of segments and tally them. A rhythm in every segment, whatever the weather and the light, is most likely the camera's own habit; one that comes and goes with wind, sun or time of day belongs to the world. A full five-minute segment takes about a minute and 1.5 GB of memory.

mkdir -p out
for f in segments/*.mp4; do
  python3 motion.py read "$f" "out/$(basename "${f%.mp4}").wav" --motion-lens hidden
done
python3 examples/hidden-tally.py out/*.txt

Tested blind on a real windy minute from the garden camera: two copies, one with a flicker of half a brightness level out of 255 (about 1%, too small to notice) planted at a random spot and rate, the key written to a file neither judge opened until both had answered. The owner, listening to the tuned sweep, picked the right copy, the right rate setting, and low in the picture. Claude, reading the numbers and sealing the answer with a hash before the listening, named the copy, the place (lower left) and the rate (1.72 a second) exactly. The rhythm lens saw nothing, and a first listening design, loud where the picture changes most, buried the flicker under the wind; loudness by how regular a patch is made it audible. One round, so far.

On that same real footage hidden also lists about eight much fainter regular changes, in both copies alike, so they are in the original video: around 0.1 levels of colour or 0.01 pixel of shift, several at about 0.18 a second, a rate that also appears once on a calm morning. They may be the camera's own processing (for example its colour balance hunting); they are not explained yet. Building the lens, a calm clip used like an astronomer's dark frame caught three flaws in it, each fixed: wind gusts read as rhythms, the camera's uneven frame timing copying motion to mirrored rates, and a bias at the slowest rates.

The camera's own habits

A security camera is not a neutral eye. Read across fifteen five-minute segments from one NVR camera (640x480, about 7 frames a second), almost every faint rhythm the lens found turned out to be the camera's own. Learn these before taking any finding as the world's:

habit	how it shows in a report	how the report handles it
fresh picture	the whole picture faintly lifted at one rate and all its multiples, e.g. every 4 s: the camera writing a full new picture (a keyframe)	named as a comb; finds at a multiple, a half or a third of its rate say "probably the camera's"
uneven frame timing	frames 0.13 and 0.17 s apart in a pattern that repeats once a second; real motion is copied faintly to rates mirrored about it	mirrored pairs in the same place are listed last as probably the camera's
printed clock	brightness, shift and texture rhythms at 0.2 to 0.6 a second in one corner	found from its digits flipping once a second; listed last as the clock, never voiced
compression breathing	a slow rhythm (every 12 to 13 s here) in many places and channels, its speed changing a little from segment to segment; in some segments the whole picture rises and falls with it, and so does the number of bytes the camera spends per frame	not yet labelled: recognise it by those signs. Likely the encoder keeping its bit rate on target
mount sway or stabilisation	a slow shift (every 10 s here) in which the whole picture moves together	not yet labelled: a shift the whole picture shares is the camera moving
night mode	colour channels empty	said in the report; colour is left out

Two things to check on the recorder: whether the camera uses variable or constant bit rate (switching it, or the bit rate, moves or removes the breathing if it is the encoder), and how the camera is mounted.

Sensitivity depends on the rate. On this camera the half-level flicker above, at 1.7 a second, was found with a chance of luck too small to print; a flicker planted at one every 12 s was missed at ±0.2 levels and found at ±0.3: wind and light crowd the slow end. A quiet report means nothing above those sizes, not nothing at all. A phone on a tripod (30 or 60 even frames a second, lighter compression) is a much better eye than an NVR.

Hidden pictures

Pictures that people deliberately draw into a sound often sit above the default 8 kHz ceiling, and most spectrogram tools draw them on a linear frequency axis rather than imgaudio's log axis. Read the whole range at the file's own 44.1 kHz, then try both scales:

python3 imgaudio.py --sr 44100 --f-lo 40 --f-hi 20000 decode clip.wav log.png
python3 imgaudio.py --sr 44100 --f-lo 40 --f-hi 20000 --freq-scale linear decode clip.wav linear.png

A picture squashed toward the top on the log scale usually comes back in shape on the linear one. --freq-scale also works on encode, and encode and decode must use the same scale. Encode refuses an --f-hi at or above half the sample rate, since those pitches can't be represented and would fold back down as noise. The full range is slower; trim to the part you care about first.

When something is buried under louder sound, --lens reveal compares each pitch with its own usual behaviour instead of with the whole clip. Hum, drones and steady noise fade; anything rare for its pitch lights up, however quiet:

python3 imgaudio.py --lens reveal decode buried.wav look.png
python3 imgaudio.py --lens reveal --lens-params smooth=0,zmax=4 decode buried.wav sharper.png

In a test, a picture mixed 20 dB under noise and mains hum went from r = 0.09 to r = 0.33 against the original. smooth (default 1) holds shapes together through noise; zmax (default 6) sets how unusual counts as full brightness.

Once you've seen the honest version, the other lenses are mostly aesthetic — you're not undoing anything, just choosing how to render it:

for L in raw edges fractal phyllotaxis spectral; do
  python3 imgaudio.py --rows 400 --cols 800 --lens $L decode short.wav "look_$L.png"
done
montage look_*.png -tile 3x -geometry 400x -label '%f' sheet.png

Output is grayscale unless the audio was encoded with --color, or you ask for --palette octave, which colours every sound by pitch (see Colour). Or colour it afterwards with any gradient:

convert look.png -auto-level \
    \( -size 1x256 gradient:'#440154-#fde725' -rotate 90 \) -clut viridis.png

From a video

motion.py subtracts each video frame from the one before it, so everything still cancels and only motion is left, then turns that motion into sound. A still scene is silent; wind in a tree is heard as it moves through the frame.

For a fixed camera. Everything here, the motion lenses included, watches fixed places in the picture over time: the same patch always shows the same leaf or the same bit of wall, and what changes there is the reading. Physicists call this the Eulerian view, after Leonhard Euler: stand still and watch the river flow past one spot. It is also the view of the MIT work this builds on (Eulerian video magnification, 2012; the visual microphone, 2014). Use a camera on a wall, a pole or a tripod. A small bump or a gentle sway is tolerated: euler notices a jolted camera and leaves those moments out, and a rhythm the whole picture shares is named as the camera's. A camera that travels, in a vehicle or in a hand, is a different case: each patch then shows the scenery sliding past, near things faster than far ones, so no fixed patch holds still and no single correction steadies the whole frame. That needs the Lagrangian view, after Joseph-Louis Lagrange: ride along with each thing and follow it (tracking points, optical flow, rebuilding the scene's depth). imgaudio does not do that. From a moving camera, only what rides with it (a dashboard, a hood, a mirror mount) stays fixed in the frame and can be read, and it tells about the vehicle, not the world.

# real time: height in the frame becomes pitch, top = high; put it back on the video
python3 motion.py profile wind.mp4 wind.wav --mux wind-heard.mp4

# full frame to frame: each change becomes a whole small picture; still steps are skipped
python3 motion.py frames wind.mp4 wind-frames.wav

# only part of the picture, and only part of the clip
python3 motion.py profile wind.mp4 tree.wav --box 0.2,0,0.5,0.6 --start 3 --duration 10

# music from only the motion
python3 notate.py notes wind.wav wind-music.wav --voice lyre --base 60 --scale nid_qablim --grid 16

--annotate also writes NAME.annotated.png beside the motion picture (or the WAV): seconds along the bottom, height in the frame up the side, and red marks where the camera skipped frames. A bump shows as a bright stripe from top to bottom. In frames mode each block is one moment that moved, labelled with its second. The sound and the plain picture are unchanged.

python3 motion.py profile wind.mp4 wind.wav --png wind.png --annotate

profile keeps the clip's timing, so its sound can be the video's soundtrack; where across the frame something moved is not kept. frames keeps whole frames (decoding its sound gives them back, r ≈ 0.87 in testing) but runs slower than the video, about one second per moving step by default. --box takes pixels or fractions of the frame. --floor (default 0.03) is how much a pixel must change to count as motion; raise it for noisy or heavily compressed video.

On a generated 20-second test clip (two gusts in a tree, grass, a bird, still in between), every still moment came out exactly silent and each motion landed in its own band: the crown at 0.8–3.5 kHz, the grass at 80–140 Hz, the bird at 4.5–5.5 kHz. Loudness follows how much moved, so a bird is far quieter than a gust. Notes from notate.py can start up to one grid step before or after motion, because it snaps onsets to the beat grid.

Reading motion: motion lenses

motion.py read hands the frames to a motion lens (motion_lenses/), which reads the motion and writes what it found as a short text file beside the WAV. Each thing it reads becomes a voice: a gliding tone whose pitch follows it (--sound glide), or notes on an instrument each time it turns around (--sound notes, any notate.py voice and scale). It reads each frame's real time, because security cameras do not record evenly and counting frames instead invents rhythms that are not there.

python3 motion.py read wind.mp4 turns.wav                     # also writes turns.txt
python3 motion.py read wind.mp4 turns.wav --sound notes --voice harp --scale nid_qablim

The first lens, euler, reads each moving thing as turns about a pivot, after Leonhard Euler's rotation theorem (1775): however a rigid body is moved about a fixed point, the result is one turn about one axis. One camera sees the flat version, a pivot in the picture and an angle. For each mover it reports where it is, where it pivots, how much a turn explains its motion compared with a slide, how far it swings, its rhythm and when it was busiest. If most of the picture moves together for much of the clip it says the camera moved and takes that away first; if the whole picture only jumps now and then (a bump, a gust shaking the mount), it names those moments and leaves them out.

On scenes generated with known answers it found every mover's rhythm exactly, put pivots within about 5% of the picture (10% at worst) and swings within about 30%; a still scene gave nothing and a shaken camera was named as such with no movers invented. It lost a swaying branch when the camera shook as well. On a five-minute garden camera clip it found the near weed stalk, its pivot at the bottom edge where the stalk enters the picture, swinging about ±1.5° with the gusts and with no steady rhythm; the clock overlay was ignored and the treetops moved too little to register at the default 320-pixel width. A five-minute clip takes about three minutes.

The second lens, rhythm, is a rhythm map: which parts of the scene repeat their motion and how often, drawn as a small letter map (one letter per region) with a line for each region. A rhythm shared by most of the picture, still parts included, is named as the camera's or the light's (exposure, frame timing, compression, flicker) and left out. On generated scenes it found each mover's rate within 0.02 repeats a second, gave nothing for a still scene, and named a whole-picture flicker as the camera. On three five-minute garden clips (two windy, one calm) it found no steady rhythm in the scene, which agrees with euler: the stalk moves with the gusts, not like a pendulum. The only regular repeat is in the clock overlay's row. A first version reported a 16-second whole-picture rhythm and 6-to-8-second repeats around the stalk; the calm clip and a direct measurement showed both came from the lens's own filtering, and it was fixed. Each rhythm is now judged against its neighbouring rates.

Both lenses were then tested on a real bump: the camera knocked once, on a windy afternoon, about 19 seconds into a 90-second clip (the picture jumped about 4 pixels in one step). At first euler said the camera was still, because it judged over the whole clip and one jolt was outvoted, and the jolt hid every mover. It now checks step by step: it reported "jolted 1 time ... at 0:19.4 (3.9 px)", left that step out, and found the stalk and the moving shadow of branches on the ground. rhythm names the same moment. The camera's own full-picture refresh every 4 seconds changes a typical patch up to 8 times its usual amount; the bump changed it 39 times, so the bar is set at 20.

python3 motion.py read wind.mp4 rhythm.wav --motion-lens rhythm

To write your own, copy motion_lenses/_example.py; see motion_lenses/README.md for the contract.

Lenses

A lens reshapes the picture before it becomes sound, or the spectrogram after it comes back, to surface structure that raw brightness hides.

python3 imgaudio.py --help                       # lists installed lenses
python3 imgaudio.py --auto-prep --lens fractal encode img.jpg out.wav
python3 imgaudio.py --lens reveal decode recording.wav look.png

lens	surfaces
edges	contours (retinal lateral inhibition)
fractal	texture roughness — natural vs man-made
phyllotaxis	golden-angle spiral read order, center-out
reveal	what is unusual for its pitch — steady sound fades, rare events light up
spectral	2D FFT — spatial frequency and orientation

spectral has four modes via --lens-params mode=...:

mode	keeps	discards
magnitude	full centered spectrum	nothing
quadrant	one quarter (the FFT of a real image is conjugate-symmetric, so the rest is redundant)	the mirror half
radial	what scales are present	orientation
angular	what directions are present	scale

Man-made scenes are full of periodic structure — brick courses, siding, mullions — and each appears as a discrete spike. Vegetation and clouds spread diffusely. Use square --rows/--cols with this lens; the spectrum is naturally square and a 200×400 grid stretches it.

Lenses also work on decode, transforming an audio's spectrogram into abstract art. Look for lens: <name> in the output — if it's missing, it didn't fire.

Colour

The audio carries luminance only — one magnitude per frequency, and colour needs three channels. --color stores the colour in a small sidecar PNG beside the WAV and reapplies it on decode.

python3 imgaudio.py --auto-prep --color encode photo.jpg photo.wav
# writes photo.wav and photo.chroma.png (~450-700 bytes)

python3 imgaudio.py --color decode photo.wav recovered.png

Human vision has far lower colour resolution than luminance resolution, which is why JPEG and every video codec subsample chroma. A 32-pixel-wide chroma plane is near-lossless to the eye:

--color-width	sidecar size	RMSE vs original
32 (default)	~450–700 B	0.025
16	~193 B	0.029
8	~119 B	0.032

The audio is byte-identical with and without --color — the flag only adds the sidecar, so files encoded either way are interchangeable as audio. The sidecar records whether --auto-prep inverted the frame, so decode flips the recovered luminance back to match the chroma's orientation.

Octave light: one palette for every sound

A recording from anywhere else has no sidecar, so there is no colour to recover. --palette octave gives decode a palette that depends on nothing but pitch, shared by every sound:

python3 imgaudio.py --palette octave decode recording.wav look.png

Visible light spans almost exactly one octave, deep red near 400 THz to violet near 790 THz. Each row's pitch is raised by whole octaves until it lands in that octave of light, and takes its place in rainbow order. A note has the same colour in every octave and in every recording; brightness is still loudness. The twelve notes are spaced evenly around the colour wheel rather than at their true wavelength colours, which would put F#, G and G# in one red.

A sidecar still wins when --color finds one. The palette only means pitch while rows are pitches, so it suits raw, edges, fractal and reveal; spectral and phyllotaxis rearrange the picture. Encode reads brightness, so re-encode the grey picture rather than the coloured one.

Tuning systems

notate.py --base selects how detected pitches are snapped.

--base 10 (default) is twelve-tone equal temperament: pitch is 440 · 2^((n−69)/12). The twelfth root of two is irrational and terminates in no number base at all.

--base 60 is just intonation: every degree is a small whole-number ratio to the root. A fifth is 3/2, a fourth 4/3, a whole tone 9/8. Because 60 divides by 2, 3 and 5, any ratio built from those primes terminates exactly in sexagesimal (3/2 = 1;30, 4/3 = 1;20, 9/8 = 1;07,30, and the pure third 5/4 = 1;15). Mesopotamian tuning was built from cycles of fifths and fourths, so a scribe could have written every one of its ratios exactly; whether base 60 shaped the music is not known. Ratios with 7 or 11 repeat forever in base 60, so the harmonic scale's 7/4 and 11/8 are the least sexagesimal of the set.

python3 notate.py scales                                    # both tables
python3 notate.py notes in.wav out.wav --base 60 --scale babylonian
python3 notate.py notes in.wav out.wav --base 60 --scale harmonic

Scales derived from stacked fifths (babylonian, babylonian_pent) land within ~6 cents of equal temperament — cleaner intervals, but familiar. The audible difference comes from scales using higher partials: harmonic reaches ~49 cents, close to a quarter-tone, because the 7th and 11th partials have no equal-tempered equivalent at all.

Standard MIDI notes are equal-tempered integers, so --midi rounds base-60 pitches to the nearest semitone. The WAV carries the true tuning; the .mid is an approximation.

The seven Babylonian tunings

The Old Babylonian tuning tablets name seven lyre tunings and a cycle for moving between them by retuning one string at a time. --base 60 offers all seven, each built from pure fifths the way the tablets build them, so every ratio is made of 2s and 3s and terminates exactly in sexagesimal:

--scale	strings 1–7 (Kilmer)	today's mode
isartum	E F G A B C D	Phrygian
kitmum	E F♯ G A B C D	Aeolian
embubum	E F♯ G A B C♯ D	Dorian
pitum	E F♯ G♯ A B C♯ D	Mixolydian
nid_qablim	E F♯ G♯ A B C♯ D♯	Ionian (major)
nis_gabari	E F♯ G♯ A♯ B C♯ D♯	Lydian
qablitum	E♯ F♯ G♯ A♯ B C♯ D♯	Locrian

Each step down the table raises one note a semitone, as the retuning texts describe. nid_qablim is identical to babylonian, and it is the tuning the Hymn to Nikkal's colophon names: notate.py scores examples/h6-together.wav highest in it (0.564, falling off around the cycle). The names follow Kilmer's reading of the strings as rising, from Crickmore (2008). West (1994) reads them as falling, which keeps the same seven patterns but moves the names: on his reading isartum is the major pattern. Interval sizes are a reconstruction too; the tablets tune by ear and give no numbers.

python3 notate.py notes in.wav out.wav --base 60 --scale nid_qablim --root A2

Hearing the difference

./examples/tuning-demo.sh input.jpg
for f in tuning_demo/*.wav; do echo "$f"; play -q "$f"; done

Five transcriptions of one source with everything held constant except the tuning. 10_minor_pent and 60_babylonian should sound nearly identical; 60_harmonic should sound audibly other. The script prints each one's --verify fidelity so you can compare what the metric says against what you hear. They don't always agree.

Instruments

--voice picks the instrument notate.py plays its notes on. They are synthesized, not recorded, but most are modelled on how the real instrument makes its sound, so each has the character of the real thing rather than of a recipe:

--voice	how it is made	MIDI instrument
vibes	the original struck tone (default; output unchanged)	11 vibraphone
vibraphone	a tuned aluminium bar (modes at 1 : 4 : 10), resonator, motor tremolo	11 vibraphone
piano	stiff strings (overtones run sharp), two or three slightly detuned strings per note, a felt hammer that is darker when played softly, a fast then slow decay, a hammer knock	0 piano
harp	a plucked string: a finger a third of the way along it, long-ringing in the bass, through a soundboard	46 harp
pad	slow swell, three detuned voices, held	89 pad
lyre	the sammûm of the Hurrian hymns and the Lyres of Ur: gut strings struck with a plectrum near the bridge, bright and short-lived, over a small wooden box	107 koto
reed	the embūbum, reed pipe, like the silver pipes of Ur: a blown double reed whose pitch settles from slightly flat, with growing vibrato, a breath waver and breath noise	111 shanai

The plucked strings are a loop one vibration long that feeds back on itself, losing a little energy each trip (more for the upper partials), started from the shape a finger or plectrum leaves. That is why higher notes and higher overtones die sooner, as on a real string. The tuning is exact (within a cent, or 0.0 cents for the strings), so the base-60 scales keep their small differences.

General MIDI has no ancient instruments, so --midi files ask for the nearest one; the WAV is the synthesized original. --program overrides it.

python3 notate.py notes examples/h6-together.wav h6-lyre.wav --voice lyre --base 60 --scale nid_qablim --root A2

Keeping the source underneath

notate.py discards the source entirely — it extracts pitch events and synthesizes fresh tones, so an image encoded into the input is gone by the time notes come out. Decoding a notate.py output shows a picture of a vibraphone performance, not of a photograph.

--dry-mix layers the original underneath the notes instead:

python3 notate.py notes src.wav out.wav --dry-mix 0.4

It recovers nothing — but the source still carries whatever encoded it, so decoding the result partly shows the picture again. Correlation with the original photo rises from 0.376 at --dry-mix 0.0 to 0.501 at 0.6. Around 0.3–0.5 gives an audible melodic line over a legible image.

for M in 0.0 0.3 0.6; do
  python3 notate.py notes src.wav "dry$M.wav" --dry-mix $M
  python3 imgaudio.py --rows 400 --cols 800 decode "dry$M.wav" "dry$M.png"
done
montage dry*.png -tile 1x -geometry 800x -label '%f' compare.png

--verify measures the pure transcription rather than the blend, so the fidelity number stays honest regardless of the mix.

A residual sidecar (source minus synthesis, saved for later re-addition) was considered and rejected. It reconstructs exactly, but the best-fit scale for the synthesis is ~0 — the notes match the source spectrally while their phases are unrelated. The residual therefore carries 100% of the source's energy and is the same size as just keeping the original.

Measuring what survives

Transcription is lossy by design. Scale snapping, grid quantization, the --voices cap, and note merging each throw away information in exchange for musicality. --verify measures how much is left.

python3 notate.py notes in.wav out.wav --verify

It re-synthesizes the transcription and correlates its log-band spectrogram against the source's:

  --- verification ---
  spectral fidelity   0.747   (1.0 identical, 0.04 noise, 0.0 silence)
  distinct pitches    30 of 392 notes
  note density        24.5 notes/sec
  pitch residual      mean  18.6 cents, max  48.7
  alphabet size       30 symbols -> 4.9 bits per note

The metric reads 1.0 for identical signals, ~0.04 for white noise, 0.0 for silence, and degrades monotonically as noise is added. It turns parameter choice from taste into measurement — at least for the question of how much gets through. Taste still governs which one sounds good.

--transcribe-opts key=val,... is an extension point for future options, same shape as --lens-params.

Findings

Measured with --verify. Read the caveats — most of this is n=1.

Holds up. Base 60 beat base 10 in all four spectral-lens modes (quadrant, angular, magnitude, radial), on both fidelity and alphabet size. Four independent comparisons, consistent direction. Ratio snapping offers more distinct landing places than twelve equal-tempered pitch classes, so fewer peaks collide onto the same symbol.

Dominates everything else: source type. Spectral-lens drone scores ~0.74; a field recording of frogs scores ~0.38. Which scale you pick is worth ~0.02. What you feed it is worth ~0.35. Broadband, inharmonic material has no clear fundamental for the peak picker to lock onto — the frog recording yielded only 8 distinct pitches from 83 notes.

Does not replicate. On the original source, harmonic (0.737) beat just_major (0.721) and babylonian (0.717). Retested on four other files, harmonic won 2 of 4 with margins under 0.022 — within noise. The mechanism is appealing (additive synthesis puts energy at harmonic partials; the harmonic scale's degrees are harmonic partials) but a plausible mechanism is a story, not evidence.

Untested beyond n=1. On one 20-second source, fidelity peaked at --voices 7 and --grid 16. Voices and grid proved separable — adding voices cost a flat ~0.010 at every grid value. Grid 12 tied grid 8 at 43% higher note density, which is the only place the sexagesimal argument extended from pitch into rhythm. All of this could be an artifact of that file's length and content.

Two operating points from that single sweep, offered as starting guesses rather than recommendations:

# best reproduction on the file tested — 0.747, 30 pitches, 24.5 notes/sec
python3 imgaudio.py --auto-prep --lens spectral --lens-params mode=quadrant \
    --rows 400 --cols 400 encode photo.jpg spec.wav
python3 notate.py notes spec.wav out.wav \
    --base 60 --scale harmonic --voices 7 --grid 16 --verify

# more throughput for 0.010 fidelity — 36 pitches, 32.8 notes/sec
python3 notate.py notes spec.wav out.wav \
    --base 60 --scale harmonic --voices 17 --grid 16 --verify

Baseline for comparison: raw drone, base 10, minor_pent, defaults → 0.470.

Full workflow: a year of frames

1. Collect frames. Name them so alphabetical = chronological: frames/image-YYYYMMDD-HHMMSS.jpg. Every batch script relies on this.

Start hourly (8,760 frames/year, ~3 days encode) rather than per-minute (525,600 frames, months). Re-run finer later if it's worth it.

2. Check settings on one frame.

python3 imgaudio.py --auto-prep roundtrip frames/first.jpg --out-dir check/

Open check/first_roundtrip.png. Middle panel should resemble the top panel.

--auto-prep is not optional for outdoor photos — it percentile-clips each frame and inverts if still mostly bright. Without it, blown-out midday frames encode as a constant chord.

3. Batch encode.

./imgaudio-batch.sh

Resumable — skips frames already done. Parallelize:

ls frames/*.jpg | parallel -j $(nproc) --bar \
  'test -f out/{/.}.wav || python3 imgaudio.py --auto-prep --rows 300 --cols 600 encode {} out/{/.}.wav'

4. Optional — lenses. Sonify hidden structure instead of raw brightness.

python3 imgaudio.py --help                       # lists installed lenses
python3 imgaudio.py --auto-prep --lens fractal encode img.jpg out.wav

See Lenses for what each one surfaces.

5. Optional — motion. Join the frames into a video and hear only what changed between them:

ffmpeg -framerate 15 -pattern_type glob -i 'frames/*.jpg' -c:v libx264 -pix_fmt yuv420p year.mp4
python3 motion.py profile year.mp4 motion.wav

Static regions go silent; movement goes loud. Light changes too, so dawn and dusk sound like motion; raise --floor to quieten slow drifts. See From a video.

6. Optional — find what changed.

./imgaudio-diff-batch.sh          # writes change_log.txt, ranked by RMS
python3 anomaly.py recurrence out/img.wav rp.png --discords 15

Encode with --no-normalize for diffs to cancel cleanly.

Recurrence plots work best one per frame, not one per year — a year at --size 1000 averages each pixel over hours of drone and comes out gray.

for f in out/*.wav; do
  python3 anomaly.py --size 600 --bands 128 recurrence "$f" "rp/$(basename "$f" .wav).png"
done
montage rp/*.png -tile 24x -geometry 80x rp/grid.png

7. Assemble and notate.

sox out/*.wav year.wav norm -3

python3 notate.py notes year.wav year_music.wav \
    --scale minor_pent --root A2 --bpm 70 --grid 4 --voices 2 \
    --midi year_music.mid

ffmpeg -i year_music.wav -c:a libopus -b:a 96k year_music.opus

For very large sets, sox out/*.wav overflows the argument list — use ls out/*.wav | sort | sed 's|^|file |' > list.txt then ffmpeg -f concat -safe 0 -i list.txt -c copy year.wav.

Rendering a recipe

render.py runs the whole pipeline from a small TOML recipe, so you don't have to remember flags or keep encode and decode settings in step by hand.

python3 render.py recipes/example.toml --dry-run    # see the plan first
python3 render.py recipes/example.toml              # render it

Outputs land in renders/<recipe name>/, one set of files per source:

file	what it is
NAME.wav	the encoded (or converted) audio
NAME.chroma.png	colour sidecar, with color = true
NAME.png	decode — the photo round-trip, or a recording's spectrogram
NAME.notes.wav / .mid	the transcription
NAME.notes.png	decode of the transcription — a picture of the performance
NAME.verify.json	fidelity figures for this source
report.csv	one row per source, with a summary printed at the end
recipe.toml	a copy of the recipe, so the piece ships with how it was made

Sections you write are stages you get: encode and decode always run; notate runs when [notate] is present; assembly runs when [assemble] is present. Any stage can be switched off with enabled = false. Recipe keys are the tools' own flags with underscores (--auto-prep becomes auto_prep), and an unknown key is an error, so a typo can't silently fall back to a default.

Resumable and safe to interrupt. Finished outputs are skipped, and each job renders into a private temp folder before moving results into place, so a killed run never leaves a half-written file behind.

Change a setting, rerun, and only what's affected is redone. Each stage's settings are hashed. Change [notate] and the encodes are kept; change [encode] and everything reruns.

Recordings as sources

Any sound file ffmpeg can read works as a source: WAV, MP3, M4A (phone recordings), FLAC, OGG. There's no hidden photo in an ordinary recording, so decode shows the sound's own structure.

[source]
audio = "recordings/*"

[decode]
palette = "magma"

• The frequency band is detected automatically from where the recording's energy actually sits, so a birdsong fills the frame instead of occupying a thin strip. Set [audio] f_lo and f_hi to fix it instead.
• Long recordings are split into equal tiles, 30 seconds by default ([audio] tile), named NAME.t001, NAME.t002, ... and laid out in reading order in NAME.sheet.png.
• Colour comes from a palette — viridis, magma, inferno, or a list of hex colours — since there's no photo to take it from.

recipes/recordings.toml is a commented starting point. Images and recordings can be mixed in one recipe.

Sweeps — trying many settings at once

render.py normally applies one configuration to many sources. A [sweep] section does the transpose: every combination of the listed values, over the same sources, ranked by how much of each source survived transcription.

[sweep.encode]
lens = ["raw", "spectral"]

[sweep.notate]
base   = [10, 60]
scale  = ["minor_pent", "harmonic"]
voices = [3, 7]

python3 render.py recipes/sweep.toml --dry-run   # count combinations first
python3 render.py recipes/sweep.toml

Each combination gets its own folder (v001, v002, ...) alongside variants.json recording what each one was, and sweep.csv with every row from every variant. At the end it prints a ranking:

sweep.csv: 36 rows, 12 variants scored
  ranked by mean fidelity across sources (n = sources scored per variant)
  0.591 ±0.024  n=3   v008  lens=raw  base=60  scale=harmonic  voices=7
  0.579 ±0.015  n=3   v007  lens=raw  base=60  scale=harmonic  voices=3
  ...
  best - worst = 0.108; per-source spread within the best variant is ±0.024

That last line is the guard against reading too much into a sweep: if the gap between best and worst is smaller than the variation between sources within one variant, it says so, and the ranking is noise.

• Combinations multiply. Four axes of two values each is sixteen variants; on twelve images that's 192 renders. --dry-run counts them, and prints the first few, before you commit.
• Encodes are shared. Variants with identical encode settings share one encode folder, so a photo is encoded once per lens rather than once per combination.
• Impossible combinations are skipped with a note — a base-60 scale name paired with base 10, for instance. Scale names are read from notate.py itself, so adding one there needs no change here.
• Sweeps don't assemble. One joined file per variant is rarely what you want; [assemble] is ignored.
• Unscored axes still work. Sweeping only lens or palette with no [notate] section produces a gallery of variant folders and says there was nothing to rank.

recipes/sweep.toml is a commented starting point.

Across a fleet

With the output folder on shared storage:

python3 render.py recipes/year.toml --prepare        # once, from one machine
python3 render.py recipes/year.toml --shard 3/40     # on each worker
python3 render.py recipes/year.toml                  # once more, to assemble

--prepare clears stale outputs and analyzes every recording once, so workers don't each repeat it. Workers take interleaved slices, so each gets a spread across the whole timeline. A worker refuses to start if the recipe changed since the last --prepare, rather than deleting files other workers may be using.

render.py gotchas

• Colour only lines up with lenses that keep things where they were: raw, edges, fractal. With spectral or phyllotaxis the decode has a different layout from the photo, so the colour tints rather than recovers. render.py warns when you combine them.
• Each tile of a recording is brightness-normalized on its own, so loudness differences between tiles don't show in the contact sheet.
• A dark frame or silent tile has nothing to transcribe. It becomes silence of the right length rather than a failure, so the assembled timeline keeps its shape.
• Assembling from = "source" with both images and recordings needs matching sample rates: set [audio] sr to match [encode] sr (both default to 22050).
• A hard kill (power loss, kill -9) can leave a stray folder in .tmp/. It's harmless, and --prepare sweeps it up.

Key parameters

imgaudio.py

flag	default	effect
--auto-prep	off	adaptive exposure handling — use it
--rows / --cols	200 / 400	vertical / horizontal resolution
--col-sec	0.05	seconds per column
--f-lo / --f-hi	80 / 8000	frequency band, Hz
--gamma	1.7	contrast; higher = starker
--color	off	write/read a chroma sidecar so decode returns colour
--color-width	32	chroma sidecar width; 8–32 all look reasonable
--palette	grey	decode only: octave colours each pitch, shared by every sound
--freq-scale	log	linear spaces rows evenly in Hz, as most spectrogram tools draw; encode and decode must match
--annotate	off	decode only: also write NAME.annotated.png with seconds, Hz and note-name rulers
--annotate-text	off	decode only: also write NAME.annotated.txt, the notes in plain text (start, name, Hz, cents) and a line on the sound's character
--lossless	off	byte-exact archive; audio is harsh noise
--no-normalize	off	needed for clean diffs
--lens / --lens-params	raw	apply a lens

notate.py

flag	default	effect
--base	10	10 = equal temperament, 60 = just intonation
--scale / --root	minor_pent / A2	run notate.py scales for both tables
--bpm / --grid	100 / 8	tempo; 4=quarters, 8=eighths, 16=sixteenths; 3/6/12 give triplets
--voices	3	max simultaneous notes
--peak-rel	0.25	density dial; lower = many more notes
--decay	2.5	lower = longer sustain
--voice	vibes	instrument for the WAV: vibes, vibraphone, piano, harp, pad, lyre, reed (see Instruments)
--program	from voice	GM instrument for --midi; defaults to the voice's nearest
--dry-mix	0.0	blend this much source back under the notes
--verify	off	report how much of the source survived
--transcribe-opts	—	key=value extension point

motion.py

flag	default	effect
mode	—	profile (real time, height = pitch), frames (each change a whole picture), read (a motion lens reads it; writes NAME.txt) or listen (tuning dial; writes NAME.txt)
--fps	15 / 5	frames per second sampled (profile / frames)
--box	whole frame	x,y,w,h in pixels or fractions; only that region
--start / --duration	whole clip	time range, seconds
--floor	0.03	brightness change that counts as motion; raise for noisy video
--width / --slot	48 / 1.0	frames mode: columns and seconds per moving step
--mux	—	profile and read modes: write the clip with this sound as its soundtrack
--annotate	off	profile and frames modes: also write NAME.annotated.png with seconds, height in the frame and skipped frames
--motion-lens	euler	read mode: which motion lens reads the motion
--motion-params	—	read mode: key=value settings for the motion lens
--sound	glide	read mode: glide (pitch follows each track) or notes (a note at each turn)
--voice / --scale / --root	harp / nid_qablim / 220	read mode, notes: instrument, scale and first note
--read-width	320	read and listen modes: frame width read at
--channel	brightness	listen mode: brightness, blue-yellow, red-green, shift-across, shift-up-down, texture
--tune / --sweep / --rates	1.0 / off / —	listen mode: one rate, or each of several in turn
--quiet / / /

anomaly.py

flag	default	effect
--size	1000	frames per axis; matrix is N², so 4000 needs ~250 MB
--bands	64	spectral bands per frame vector; higher sharpens striping
--mode	continuous	binary with --percentile gives the stark print look
--discords	0	also print the N most anomalous timestamps

Examples

examples/tuning-demo.sh IMAGE — five transcriptions of one source with only the tuning varying, each with its --verify score. The fastest way to hear what --base 60 means.

examples/voice-stack.sh INPUT.wav [OUT_DIR] [MAX_PRIME] [--layer] — transcribes one file at each prime --voices value, then joins them. Sequential by default (sparse to dense, the progression audible as form); --layer mixes them simultaneously instead. Caps at 40 because note count peaks near --voices 37 and declines after — low-amplitude peaks jitter between neighbouring bins and merge into fewer, longer notes rather than more of them.

examples/hidden-word.wav and examples/hidden-stone.wav — test clips with pictures hidden the way musicians hide them (see Hidden pictures). The word "imgaudio" sits at 10–18 kHz under a melody, out of sight until you read the full range; the stone is drawn on a linear frequency scale and only comes back in shape on one. Both are 44.1 kHz, so they also work in the web page's Sound mode with Frequency range set to Full. examples/hidden-pictures.py rebuilds them and shows exactly how they were made.

examples/h6-together.wav and examples/h6-in-turn.wav — the opening of the Hymn to Nikkal (Hurrian hymn h.6, Ugarit, about 1400 BCE), the oldest substantially complete written music known, rendered from the tablet's own notation: names of lyre string pairs, each with a number. The lyre is tuned in nid_qablim, the tuning the tablet's colophon names: the same stacked fifths and fourths the Old Babylonian tuning tablets describe. Scholars disagree on whether each pair sounded together (Kilmer) or in turn (Dumbrill), so there is one clip of each. Only the first two notation lines are included, as transcribed from Laroche, Ugaritica V (1968); examples/hurrian-h6.py lists exactly what comes from the sources and what is a choice, and new lines can be added to its NOTATION list. See them with python3 imgaudio.py --palette octave --f-lo 100 --f-hi 1600 decode examples/h6-together.wav h6.png.

examples/seikilos.wav — the Seikilos epitaph (1st or 2nd century CE), the oldest complete piece of music whose melody and rhythm can both be read. The melody comes straight from the letters carved over each syllable, with their length marks, played in Pythagorean tuning. With --base 60 --scale nid_qablim notate.py hears it back at a fidelity near 0.9, since stacked fifths are the song's own tuning. examples/seikilos.py gives the sources and the one place it follows scholars over the image (the final cadence).

examples/enkidu-clay.wav and examples/enkidu-beasts.wav — lines from the Epic of Gilgamesh about Enkidu, shaped from clay and living with the beasts, in R. Campbell Thompson's public-domain 1928 translation, written into sound. Decode them and the words come back: python3 imgaudio.py --rows 300 --cols 600 decode examples/enkidu-clay.wav clay.png. What travels is the shape of the writing, not speech. examples/enkidu.py keeps the text in the upper two thirds of the picture, because below about 300 Hz the decoder resolves pitch too coarsely and letters smear.

Writing a lens

Copy lenses/_example.py, rename, implement three things:

NAME = "my_lens"
DESCRIPTION = "one line, shown in --help"

def analyze(grid, params):
    # grid: (rows, cols) float32 in [0,1]; params: dict[str,str]
    return new_grid

Auto-discovered, no registration. Two rules learned the hard way:

• Filename must match NAME — imgaudio.py registers by NAME, lens_recursion.sh discovers by filename.
• Keep output dense — a lens producing >90% zeros collapses to silence after two or three recursions. This is why phyllotaxis smears each spiral sample across rows with a Gaussian instead of placing a single bright pixel.

The web page

web/ is the browser version. It runs the repository's own imgaudio.py, notate.py and lenses/ unchanged, in Python compiled to WebAssembly (Pyodide), inside a background worker. There is no server: GitHub Pages serves static files, and everything runs on the visitor's device.

file	role
index.html	the page: Photo and Sound modes, settings, results
worker.js	loads Pyodide and the scripts, runs each request off the main thread
bridge.py	turns the page's settings into the scripts' own command-line flags; also swaps in a scipy STFT helper that fits 32-bit WebAssembly (identical numbers)
build.sh	assembles the site into _site/
icons/, manifest.webmanifest	tab and home-screen icons: the translators' stone

Adding a lens or a scale to the scripts needs no change here; the page reads them at startup. Only the grouping and display names of the scale menus live in bridge.py, and anything not listed there still appears under "Other".

Try it locally (no virtual environment needed):

./web/build.sh
python3 -m http.server -d _site 8000      # then open http://localhost:8000

Rebuild after changes and hard-refresh the browser so it picks up the new worker.

Deploying is automatic: every push to main runs .github/workflows/pages.yml, which builds and publishes the site. The one-time setup is Settings → Pages → Source: GitHub Actions.

Versions. VERSION holds the release number and is bumped by hand (echo 0.7.0 > VERSION). The build adds the commit and date, marking a build from uncommitted changes -dirty, and the footer shows all three. Its "Report a problem" link opens a GitHub issue pre-filled with the version, the browser and the last error.

Gotchas

• --lossless ignores lenses, colour, and all spectrogram parameters. It's a byte copy — the original file already has its colour.
• Encode and decode must use the same --rows, --cols, --f-lo, --f-hi and --freq-scale.
• A lens during encode means the round-trip reconstructs what the lens saw, not the original. That's the point, but it surprises you the first time.
• notate.py output cannot be decoded back to the source picture. It's a branch off the pipeline, not a stage in it — the notes are synthesized from scratch and the source is gone; decoding it shows the notes. Use --dry-mix if you need the image to survive.
• --color on decode without a matching .chroma.png warns and falls back to grayscale. The sidecar must sit beside the WAV with the same stem, so move or copy them together.
• --base 60 with a base-10 scale name exits with "unknown scale". The default minor_pent is auto-swapped for babylonian_pent, but other base-10 names will fail — run notate.py scales to see both tables.
• The spectral lens output is conjugate-symmetric, so its audio has partials in mirrored pairs and sounds unusually consonant for reasons that have nothing to do with the picture. Use mode=quadrant to break the symmetry.
• matplotlib: divide by zero in log10 on silent bins — harmless.
• Badly clipped sources need help beyond --auto-prep: convert in.jpg -colorspace Gray -negate -level 0%,30%,1.2 prepped.jpg
• Run anomaly.py discords against motion audio, not brightness audio — a year of drone is so self-similar that "most anomalous" just means "loudest."
• Transcribing notate.py output scores deceptively high (~0.89). The source is already sparse, pitched, and quantized, so --verify measures self-consistency rather than channel capacity. Test on unfamiliar material.
• voice-stack.sh takes a .wav, not an image, and its second argument is the output directory rather than the prime cap. Both mistakes now error out rather than failing silently.

The tradeoff

Raw bytes: perfect reconstruction, unlistenable noise. Pentatonic mapping: music, no reconstruction. Spectrogram synthesis sits between, and every lens trades a little more fidelity for a little more meaning.

Optimizing for --verify fidelity and optimizing for something worth hearing are different objectives. just_pent scores worst of the base-60 scales (0.657) despite being the prettiest — five degrees isn't enough resolution to carry a signal. The metric tells you what got through, not whether it was worth transmitting.

Acknowledgements

Built in collaboration with Claude (Anthropic), which wrote much of the code and helped test it. The ideas and direction — the year of frames, lenses as nature's pattern-finders, base-60 tuning, keeping the source underneath the notes, the colour sidecar, and the Chromebook render farm — came from the author, as did the judgement about which results actually held up.