Desktop companion robot / Raspberry Pi 4
Actual frame from the renderer. Each eye and the mouth are separately addressable.
Theo already had a face, a voice and a camera. Party mode gives him a fourth sense: he joins your lighting rig as a device, watches you nod, works out the tempo, and keeps the groove going when you stop.
00 — See it running
Recorded off the Pi with OpenRGB driving the face. Sound is part of it — the music is what the lighting visualiser is reacting to, and the beat you hear is the beat Theo is recovering from colour.
Everything below is how each of those was built, and what measurement said about it.
01 — He is an OpenRGB device
E1.31 (sACN) is how lighting software talks to network fixtures — plain UDP with a 126-byte header, so no library, no dependency. Theo appears in OpenRGB as a real device: a 3 × 10 matrix whose three columns are his left eye, right eye and mouth.
A separate process on its own core. The render loop is never allowed to touch the network.
OpenRGB sends no audio. But when its visualiser is running the beat is already in the colour stream, so the bounce is recovered from brightness rising above its own recent average — not invented. A static colour produces no bounce, which is the honest answer rather than a bug.
02 — He watches you
The first version tracked the bounding box around your head and hardly reacted. Measured live, a nod moves the box centre through 2.6% of the frame while head pitch swings whole degrees. Reading pose instead — as deviation from a slowly adapting baseline, because a camera on a desk sees you at a permanent +30° — produced 8.5× the response from the same nod.
He holds eye contact while dancing — the social gaze model that makes him look away during conversation is suppressed. And because he has no neck, a sustained offset migrates out of his eyes into a whole-face lean over two seconds, the way a person's head takes over from their eyes.
03 — He keeps the beat
Beats come from upward zero crossings of the nod signal — a peak is one noisy sample, a sign change is not. Once three intervals agree he locks a tempo and phase-locks an oscillator to you. When you ease off, that oscillator fades in underneath your own movement, already in phase, so there is no seam to see.
He hands control back over six beats rather than snapping to you — he already has your tempo, so there is no hurry. Every 20–40 seconds he takes a turn of his own even while you are still dancing. And he keeps re-learning your tempo, depth and phase the entire time he is the one leading.
04 — What measurement found
dt × ω < 1.5.
Party mode's stiffer springs pushed it to 2.30, so a single slow frame overshot to
3× the target and rang. Latent in every mode; party mode made it loud enough
to see.
One hitched frame: 3.14× overshoot → 0.9905 — The rest of the time
Outside party mode the same pixels report how he is doing: hue carries temperature, colour drains toward red if frames slip, and an undervoltage event browns the whole face out. While OpenRGB is driving, all of that stands down — it owns the colour outright.